Network is under initialization...
Network successfully initialized.
INFO: Downloading File to /root/PM-DARTS2/...

Succeed: Total num: 40, size: 339,637,308. OK num: 40(download 40 objects).

average speed 369975000(byte/s)

0.921500(s) elapsed
INFO: Downloading succeed.
WARN: ./requirements.txt not found, skip installing requirements.
Training with a single process on 1 GPUs.
Data processing configuration for current model + dataset:
	input_size: (3, 32, 32)
	interpolation: bilinear
	mean: (0.49139968, 0.48215827, 0.44653124)
	std: (0.24703233, 0.24348505, 0.26158768)
	crop_pct: 1.0
	crop_mode: center

-------------------------------
Learnable parameters
Student: 0.69M
Extra: 0.00M
-------------------------------
Scheduled epochs: 50
p_max: 0.2
search_space = s2
Using downloaded and verified file: /mnt/PM-DARTS2/data/cifar-100-python.tar.gz
Extracting /mnt/PM-DARTS2/data/cifar-100-python.tar.gz to /mnt/PM-DARTS2/data
Train: 0 [   0/390]  Loss: 4.586 (4.59)  Acc@1:  3.1250 ( 3.1250)  Acc@5:  9.3750 ( 9.3750)LR: 2.500e-02
Train: 0 [  50/390]  Loss: 4.121 (4.45)  Acc@1:  7.8125 ( 3.3701)  Acc@5: 23.4375 (12.5000)LR: 2.500e-02
Train: 0 [ 100/390]  Loss: 4.260 (4.35)  Acc@1:  4.6875 ( 4.5947)  Acc@5: 12.5000 (15.9189)LR: 2.500e-02
Train: 0 [ 150/390]  Loss: 3.776 (4.26)  Acc@1: 10.9375 ( 5.4843)  Acc@5: 29.6875 (19.0294)LR: 2.500e-02
Train: 0 [ 200/390]  Loss: 3.769 (4.17)  Acc@1:  9.3750 ( 6.5299)  Acc@5: 31.2500 (22.0460)LR: 2.500e-02
Train: 0 [ 250/390]  Loss: 3.506 (4.10)  Acc@1:  6.2500 ( 7.4639)  Acc@5: 46.8750 (24.3837)LR: 2.500e-02
Train: 0 [ 300/390]  Loss: 3.812 (4.04)  Acc@1: 15.6250 ( 8.1707)  Acc@5: 37.5000 (26.2614)LR: 2.500e-02
Train: 0 [ 350/390]  Loss: 3.638 (3.98)  Acc@1: 12.5000 ( 8.8942)  Acc@5: 37.5000 (27.9870)LR: 2.500e-02
Train: 0 [ 390/390]  Loss: 3.613 (3.94)  Acc@1: 17.5000 ( 9.4880)  Acc@5: 40.0000 (29.1520)LR: 2.500e-02
train_acc 9.488000
Valid: 0 [   0/390]  Loss: 3.639 (3.64)  Acc@1: 20.3125 (20.3125)  Acc@5: 43.7500 (43.7500)
Valid: 0 [  50/390]  Loss: 3.486 (3.63)  Acc@1: 18.7500 (14.4914)  Acc@5: 40.6250 (39.9203)
Valid: 0 [ 100/390]  Loss: 3.555 (3.63)  Acc@1: 12.5000 (14.4802)  Acc@5: 42.1875 (39.5266)
Valid: 0 [ 150/390]  Loss: 3.598 (3.63)  Acc@1: 14.0625 (14.3522)  Acc@5: 39.0625 (39.4247)
Valid: 0 [ 200/390]  Loss: 3.716 (3.62)  Acc@1: 12.5000 (14.6999)  Acc@5: 35.9375 (39.7155)
Valid: 0 [ 250/390]  Loss: 3.897 (3.61)  Acc@1: 10.9375 (14.8282)  Acc@5: 34.3750 (39.8967)
Valid: 0 [ 300/390]  Loss: 3.450 (3.60)  Acc@1: 20.3125 (14.8619)  Acc@5: 50.0000 (40.1007)
Valid: 0 [ 350/390]  Loss: 3.668 (3.61)  Acc@1: 17.1875 (14.7302)  Acc@5: 43.7500 (39.9127)
Valid: 0 [ 390/390]  Loss: 3.330 (3.61)  Acc@1: 12.5000 (14.6480)  Acc@5: 47.5000 (39.9240)
valid_acc 14.648000
epoch = 0   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4965, 0.5035],
        [0.4936, 0.5064],
        [0.4911, 0.5089],
        [0.4903, 0.5097],
        [0.4925, 0.5075],
        [0.4973, 0.5027],
        [0.4882, 0.5118],
        [0.4880, 0.5120],
        [0.4850, 0.5150],
        [0.4963, 0.5037],
        [0.4883, 0.5117],
        [0.4898, 0.5102],
        [0.4905, 0.5095],
        [0.4874, 0.5126]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4954, 0.5046],
        [0.4925, 0.5075],
        [0.5011, 0.4989],
        [0.4985, 0.5015],
        [0.4923, 0.5077],
        [0.4935, 0.5065],
        [0.4926, 0.5074],
        [0.4901, 0.5099],
        [0.4935, 0.5065],
        [0.4980, 0.5020],
        [0.4934, 0.5066],
        [0.4958, 0.5042],
        [0.4970, 0.5030],
        [0.4956, 0.5044]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 1 [   0/390]  Loss: 3.653 (3.65)  Acc@1: 10.9375 (10.9375)  Acc@5: 40.6250 (40.6250)LR: 2.498e-02
Train: 1 [  50/390]  Loss: 3.459 (3.50)  Acc@1: 21.8750 (15.9620)  Acc@5: 48.4375 (43.6887)LR: 2.498e-02
Train: 1 [ 100/390]  Loss: 3.420 (3.48)  Acc@1: 18.7500 (16.3366)  Acc@5: 46.8750 (43.8892)LR: 2.498e-02
Train: 1 [ 150/390]  Loss: 3.970 (3.44)  Acc@1:  9.3750 (16.6805)  Acc@5: 37.5000 (44.8158)LR: 2.498e-02
Train: 1 [ 200/390]  Loss: 3.260 (3.44)  Acc@1: 26.5625 (16.5812)  Acc@5: 50.0000 (44.7606)LR: 2.498e-02
Train: 1 [ 250/390]  Loss: 3.264 (3.40)  Acc@1: 15.6250 (17.1252)  Acc@5: 50.0000 (45.5926)LR: 2.498e-02
Train: 1 [ 300/390]  Loss: 3.314 (3.37)  Acc@1: 20.3125 (17.8935)  Acc@5: 45.3125 (46.2780)LR: 2.498e-02
Train: 1 [ 350/390]  Loss: 3.462 (3.35)  Acc@1: 23.4375 (18.2826)  Acc@5: 48.4375 (46.8527)LR: 2.498e-02
Train: 1 [ 390/390]  Loss: 3.043 (3.34)  Acc@1: 22.5000 (18.4960)  Acc@5: 52.5000 (47.1280)LR: 2.498e-02
train_acc 18.496000
Valid: 1 [   0/390]  Loss: 3.232 (3.23)  Acc@1: 20.3125 (20.3125)  Acc@5: 50.0000 (50.0000)
Valid: 1 [  50/390]  Loss: 3.465 (3.20)  Acc@1: 18.7500 (21.4461)  Acc@5: 48.4375 (51.2561)
Valid: 1 [ 100/390]  Loss: 3.368 (3.21)  Acc@1: 21.8750 (21.0860)  Acc@5: 53.1250 (51.4697)
Valid: 1 [ 150/390]  Loss: 3.065 (3.21)  Acc@1: 28.1250 (21.2541)  Acc@5: 64.0625 (51.3659)
Valid: 1 [ 200/390]  Loss: 3.388 (3.22)  Acc@1: 15.6250 (21.2220)  Acc@5: 39.0625 (50.8940)
Valid: 1 [ 250/390]  Loss: 2.876 (3.23)  Acc@1: 26.5625 (21.1467)  Acc@5: 54.6875 (50.5976)
Valid: 1 [ 300/390]  Loss: 3.352 (3.24)  Acc@1: 25.0000 (21.2417)  Acc@5: 45.3125 (50.3789)
Valid: 1 [ 350/390]  Loss: 3.473 (3.24)  Acc@1: 15.6250 (21.1360)  Acc@5: 51.5625 (50.3739)
Valid: 1 [ 390/390]  Loss: 3.056 (3.24)  Acc@1: 30.0000 (21.1800)  Acc@5: 52.5000 (50.5440)
valid_acc 21.180000
epoch = 1   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4898, 0.5102],
        [0.4840, 0.5160],
        [0.4796, 0.5204],
        [0.4813, 0.5187],
        [0.4857, 0.5143],
        [0.4872, 0.5128],
        [0.4771, 0.5229],
        [0.4785, 0.5215],
        [0.4793, 0.5207],
        [0.4883, 0.5117],
        [0.4769, 0.5231],
        [0.4810, 0.5190],
        [0.4825, 0.5175],
        [0.4781, 0.5219]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4918, 0.5082],
        [0.4857, 0.5143],
        [0.5011, 0.4989],
        [0.4950, 0.5050],
        [0.4781, 0.5219],
        [0.4948, 0.5052],
        [0.4810, 0.5190],
        [0.4810, 0.5190],
        [0.4850, 0.5150],
        [0.4926, 0.5074],
        [0.4898, 0.5102],
        [0.4892, 0.5108],
        [0.4864, 0.5136],
        [0.4867, 0.5133]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 2 [   0/390]  Loss: 3.113 (3.11)  Acc@1: 26.5625 (26.5625)  Acc@5: 50.0000 (50.0000)LR: 2.491e-02
Train: 2 [  50/390]  Loss: 2.889 (3.10)  Acc@1: 28.1250 (23.0086)  Acc@5: 60.9375 (53.3395)LR: 2.491e-02
Train: 2 [ 100/390]  Loss: 2.909 (3.08)  Acc@1: 39.0625 (23.6386)  Acc@5: 54.6875 (53.6355)LR: 2.491e-02
Train: 2 [ 150/390]  Loss: 2.931 (3.07)  Acc@1: 28.1250 (23.7376)  Acc@5: 62.5000 (53.8700)LR: 2.491e-02
Train: 2 [ 200/390]  Loss: 3.046 (3.06)  Acc@1: 29.6875 (23.8884)  Acc@5: 57.8125 (54.5320)LR: 2.491e-02
Train: 2 [ 250/390]  Loss: 2.927 (3.03)  Acc@1: 21.8750 (24.2966)  Acc@5: 46.8750 (55.0797)LR: 2.491e-02
Train: 2 [ 300/390]  Loss: 2.619 (3.02)  Acc@1: 32.8125 (24.4965)  Acc@5: 62.5000 (55.1235)LR: 2.491e-02
Train: 2 [ 350/390]  Loss: 2.788 (3.01)  Acc@1: 29.6875 (24.8130)  Acc@5: 67.1875 (55.4087)LR: 2.491e-02
Train: 2 [ 390/390]  Loss: 2.997 (3.00)  Acc@1: 25.0000 (24.9200)  Acc@5: 62.5000 (55.7680)LR: 2.491e-02
train_acc 24.920000
Valid: 2 [   0/390]  Loss: 3.386 (3.39)  Acc@1: 17.1875 (17.1875)  Acc@5: 48.4375 (48.4375)
Valid: 2 [  50/390]  Loss: 2.801 (2.99)  Acc@1: 31.2500 (25.4902)  Acc@5: 56.2500 (55.1471)
Valid: 2 [ 100/390]  Loss: 3.344 (3.00)  Acc@1: 18.7500 (25.1238)  Acc@5: 48.4375 (54.9814)
Valid: 2 [ 150/390]  Loss: 3.032 (2.99)  Acc@1: 23.4375 (25.4760)  Acc@5: 54.6875 (55.4946)
Valid: 2 [ 200/390]  Loss: 2.842 (3.00)  Acc@1: 31.2500 (25.4975)  Acc@5: 60.9375 (55.4493)
Valid: 2 [ 250/390]  Loss: 3.196 (3.00)  Acc@1: 21.8750 (25.1494)  Acc@5: 57.8125 (55.3038)
Valid: 2 [ 300/390]  Loss: 2.985 (3.01)  Acc@1: 21.8750 (25.0571)  Acc@5: 50.0000 (55.2066)
Valid: 2 [ 350/390]  Loss: 3.324 (3.01)  Acc@1: 23.4375 (25.2226)  Acc@5: 48.4375 (55.3730)
Valid: 2 [ 390/390]  Loss: 2.974 (3.01)  Acc@1: 27.5000 (25.2800)  Acc@5: 55.0000 (55.3240)
valid_acc 25.280000
epoch = 2   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4833, 0.5167],
        [0.4723, 0.5277],
        [0.4719, 0.5281],
        [0.4673, 0.5327],
        [0.4729, 0.5271],
        [0.4740, 0.5260],
        [0.4681, 0.5319],
        [0.4660, 0.5340],
        [0.4651, 0.5349],
        [0.4800, 0.5200],
        [0.4653, 0.5347],
        [0.4692, 0.5308],
        [0.4697, 0.5303],
        [0.4635, 0.5365]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4856, 0.5144],
        [0.4804, 0.5196],
        [0.5002, 0.4998],
        [0.4905, 0.5095],
        [0.4649, 0.5351],
        [0.4900, 0.5100],
        [0.4741, 0.5259],
        [0.4711, 0.5289],
        [0.4775, 0.5225],
        [0.4954, 0.5046],
        [0.4832, 0.5168],
        [0.4797, 0.5203],
        [0.4782, 0.5218],
        [0.4793, 0.5207]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 3 [   0/390]  Loss: 2.780 (2.78)  Acc@1: 31.2500 (31.2500)  Acc@5: 57.8125 (57.8125)LR: 2.479e-02
Train: 3 [  50/390]  Loss: 2.970 (2.80)  Acc@1: 26.5625 (28.0637)  Acc@5: 60.9375 (59.9265)LR: 2.479e-02
Train: 3 [ 100/390]  Loss: 2.747 (2.79)  Acc@1: 31.2500 (28.5736)  Acc@5: 64.0625 (60.3032)LR: 2.479e-02
Train: 3 [ 150/390]  Loss: 2.863 (2.78)  Acc@1: 26.5625 (29.1494)  Acc@5: 60.9375 (60.4512)LR: 2.479e-02
Train: 3 [ 200/390]  Loss: 2.584 (2.75)  Acc@1: 21.8750 (29.4387)  Acc@5: 71.8750 (61.4272)LR: 2.479e-02
Train: 3 [ 250/390]  Loss: 2.908 (2.75)  Acc@1: 18.7500 (29.6315)  Acc@5: 51.5625 (61.7468)LR: 2.479e-02
Train: 3 [ 300/390]  Loss: 2.724 (2.73)  Acc@1: 31.2500 (29.9990)  Acc@5: 64.0625 (62.1989)LR: 2.479e-02
Train: 3 [ 350/390]  Loss: 2.471 (2.72)  Acc@1: 35.9375 (30.1327)  Acc@5: 70.3125 (62.4955)LR: 2.479e-02
Train: 3 [ 390/390]  Loss: 3.132 (2.72)  Acc@1: 22.5000 (30.3240)  Acc@5: 55.0000 (62.6840)LR: 2.479e-02
train_acc 30.324000
Valid: 3 [   0/390]  Loss: 2.776 (2.78)  Acc@1: 31.2500 (31.2500)  Acc@5: 57.8125 (57.8125)
Valid: 3 [  50/390]  Loss: 2.534 (2.75)  Acc@1: 39.0625 (30.6373)  Acc@5: 67.1875 (63.0208)
Valid: 3 [ 100/390]  Loss: 2.662 (2.74)  Acc@1: 39.0625 (31.1417)  Acc@5: 68.7500 (63.2426)
Valid: 3 [ 150/390]  Loss: 2.665 (2.77)  Acc@1: 31.2500 (30.2670)  Acc@5: 65.6250 (62.1068)
Valid: 3 [ 200/390]  Loss: 2.424 (2.77)  Acc@1: 37.5000 (30.0995)  Acc@5: 70.3125 (61.9170)
Valid: 3 [ 250/390]  Loss: 2.830 (2.77)  Acc@1: 31.2500 (30.2913)  Acc@5: 57.8125 (61.7779)
Valid: 3 [ 300/390]  Loss: 2.608 (2.77)  Acc@1: 32.8125 (30.4454)  Acc@5: 62.5000 (61.9549)
Valid: 3 [ 350/390]  Loss: 2.808 (2.77)  Acc@1: 26.5625 (30.3241)  Acc@5: 59.3750 (61.8100)
Valid: 3 [ 390/390]  Loss: 2.463 (2.78)  Acc@1: 35.0000 (30.3800)  Acc@5: 70.0000 (61.7200)
valid_acc 30.380000
epoch = 3   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4667, 0.5333],
        [0.4593, 0.5407],
        [0.4551, 0.5449],
        [0.4557, 0.5443],
        [0.4528, 0.5472],
        [0.4557, 0.5443],
        [0.4569, 0.5431],
        [0.4477, 0.5523],
        [0.4443, 0.5557],
        [0.4679, 0.5321],
        [0.4463, 0.5537],
        [0.4507, 0.5493],
        [0.4507, 0.5493],
        [0.4477, 0.5523]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4763, 0.5237],
        [0.4762, 0.5238],
        [0.4929, 0.5071],
        [0.4848, 0.5152],
        [0.4499, 0.5501],
        [0.4828, 0.5172],
        [0.4621, 0.5379],
        [0.4553, 0.5447],
        [0.4624, 0.5376],
        [0.4916, 0.5084],
        [0.4791, 0.5209],
        [0.4656, 0.5344],
        [0.4692, 0.5308],
        [0.4730, 0.5270]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 4 [   0/390]  Loss: 2.592 (2.59)  Acc@1: 35.9375 (35.9375)  Acc@5: 67.1875 (67.1875)LR: 2.462e-02
Train: 4 [  50/390]  Loss: 2.503 (2.53)  Acc@1: 28.1250 (34.7120)  Acc@5: 57.8125 (67.1569)LR: 2.462e-02
Train: 4 [ 100/390]  Loss: 2.280 (2.55)  Acc@1: 35.9375 (34.0501)  Acc@5: 76.5625 (66.8317)LR: 2.462e-02
Train: 4 [ 150/390]  Loss: 2.445 (2.56)  Acc@1: 32.8125 (33.8576)  Acc@5: 68.7500 (66.6701)LR: 2.462e-02
Train: 4 [ 200/390]  Loss: 2.702 (2.53)  Acc@1: 29.6875 (34.2895)  Acc@5: 65.6250 (67.0476)LR: 2.462e-02
Train: 4 [ 250/390]  Loss: 2.401 (2.53)  Acc@1: 32.8125 (34.3314)  Acc@5: 68.7500 (67.1128)LR: 2.462e-02
Train: 4 [ 300/390]  Loss: 2.477 (2.51)  Acc@1: 34.3750 (34.6242)  Acc@5: 67.1875 (67.3017)LR: 2.462e-02
Train: 4 [ 350/390]  Loss: 2.733 (2.50)  Acc@1: 42.1875 (34.7356)  Acc@5: 67.1875 (67.6549)LR: 2.462e-02
Train: 4 [ 390/390]  Loss: 2.598 (2.49)  Acc@1: 35.0000 (35.0360)  Acc@5: 67.5000 (67.7760)LR: 2.462e-02
train_acc 35.036000
Valid: 4 [   0/390]  Loss: 2.554 (2.55)  Acc@1: 31.2500 (31.2500)  Acc@5: 67.1875 (67.1875)
Valid: 4 [  50/390]  Loss: 2.604 (2.51)  Acc@1: 31.2500 (35.6924)  Acc@5: 67.1875 (68.7806)
Valid: 4 [ 100/390]  Loss: 2.553 (2.48)  Acc@1: 34.3750 (35.8911)  Acc@5: 75.0000 (68.7964)
Valid: 4 [ 150/390]  Loss: 2.451 (2.49)  Acc@1: 40.6250 (35.7099)  Acc@5: 68.7500 (68.3878)
Valid: 4 [ 200/390]  Loss: 2.298 (2.50)  Acc@1: 37.5000 (35.3156)  Acc@5: 75.0000 (68.3225)
Valid: 4 [ 250/390]  Loss: 2.563 (2.51)  Acc@1: 40.6250 (35.2776)  Acc@5: 64.0625 (67.9221)
Valid: 4 [ 300/390]  Loss: 2.544 (2.51)  Acc@1: 31.2500 (35.1640)  Acc@5: 68.7500 (67.8727)
Valid: 4 [ 350/390]  Loss: 2.438 (2.51)  Acc@1: 40.6250 (35.1451)  Acc@5: 67.1875 (67.8374)
Valid: 4 [ 390/390]  Loss: 2.271 (2.51)  Acc@1: 32.5000 (35.0560)  Acc@5: 75.0000 (67.7480)
valid_acc 35.056000
epoch = 4   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4500, 0.5500],
        [0.4419, 0.5581],
        [0.4384, 0.5616],
        [0.4435, 0.5565],
        [0.4316, 0.5684],
        [0.4380, 0.5620],
        [0.4416, 0.5584],
        [0.4265, 0.5735],
        [0.4268, 0.5732],
        [0.4536, 0.5464],
        [0.4250, 0.5750],
        [0.4291, 0.5709],
        [0.4359, 0.5641],
        [0.4298, 0.5702]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4680, 0.5320],
        [0.4718, 0.5282],
        [0.4760, 0.5240],
        [0.4808, 0.5192],
        [0.4379, 0.5621],
        [0.4713, 0.5287],
        [0.4542, 0.5458],
        [0.4407, 0.5593],
        [0.4511, 0.5489],
        [0.4852, 0.5148],
        [0.4762, 0.5238],
        [0.4501, 0.5499],
        [0.4608, 0.5392],
        [0.4644, 0.5356]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 5 [   0/390]  Loss: 2.187 (2.19)  Acc@1: 42.1875 (42.1875)  Acc@5: 71.8750 (71.8750)LR: 2.441e-02
Train: 5 [  50/390]  Loss: 2.357 (2.36)  Acc@1: 35.9375 (37.4387)  Acc@5: 71.8750 (70.5270)LR: 2.441e-02
Train: 5 [ 100/390]  Loss: 2.166 (2.33)  Acc@1: 43.7500 (38.2580)  Acc@5: 75.0000 (70.7921)LR: 2.441e-02
Train: 5 [ 150/390]  Loss: 2.216 (2.34)  Acc@1: 35.9375 (38.1416)  Acc@5: 76.5625 (70.5919)LR: 2.441e-02
Train: 5 [ 200/390]  Loss: 2.339 (2.34)  Acc@1: 35.9375 (38.3318)  Acc@5: 71.8750 (70.7867)LR: 2.441e-02
Train: 5 [ 250/390]  Loss: 2.700 (2.33)  Acc@1: 35.9375 (38.2470)  Acc@5: 65.6250 (71.0035)LR: 2.441e-02
Train: 5 [ 300/390]  Loss: 2.497 (2.34)  Acc@1: 37.5000 (38.3461)  Acc@5: 70.3125 (71.0185)LR: 2.441e-02
Train: 5 [ 350/390]  Loss: 2.531 (2.33)  Acc@1: 34.3750 (38.5906)  Acc@5: 70.3125 (71.1360)LR: 2.441e-02
Train: 5 [ 390/390]  Loss: 1.862 (2.32)  Acc@1: 57.5000 (38.7760)  Acc@5: 75.0000 (71.4760)LR: 2.441e-02
train_acc 38.776000
Valid: 5 [   0/390]  Loss: 2.616 (2.62)  Acc@1: 39.0625 (39.0625)  Acc@5: 67.1875 (67.1875)
Valid: 5 [  50/390]  Loss: 2.166 (2.42)  Acc@1: 40.6250 (38.6029)  Acc@5: 73.4375 (70.6189)
Valid: 5 [ 100/390]  Loss: 2.820 (2.40)  Acc@1: 26.5625 (38.5056)  Acc@5: 62.5000 (70.5755)
Valid: 5 [ 150/390]  Loss: 2.327 (2.38)  Acc@1: 42.1875 (38.8245)  Acc@5: 70.3125 (70.5815)
Valid: 5 [ 200/390]  Loss: 2.363 (2.37)  Acc@1: 45.3125 (39.0470)  Acc@5: 73.4375 (70.7945)
Valid: 5 [ 250/390]  Loss: 2.453 (2.36)  Acc@1: 34.3750 (39.1995)  Acc@5: 75.0000 (70.8105)
Valid: 5 [ 300/390]  Loss: 2.387 (2.37)  Acc@1: 29.6875 (39.0106)  Acc@5: 67.1875 (70.7434)
Valid: 5 [ 350/390]  Loss: 2.021 (2.36)  Acc@1: 45.3125 (39.1248)  Acc@5: 75.0000 (70.9535)
Valid: 5 [ 390/390]  Loss: 2.539 (2.36)  Acc@1: 27.5000 (39.0440)  Acc@5: 62.5000 (70.8480)
valid_acc 39.044000
epoch = 5   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4358, 0.5642],
        [0.4250, 0.5750],
        [0.4247, 0.5753],
        [0.4344, 0.5656],
        [0.4127, 0.5873],
        [0.4236, 0.5764],
        [0.4310, 0.5690],
        [0.4078, 0.5922],
        [0.4085, 0.5915],
        [0.4378, 0.5622],
        [0.4093, 0.5907],
        [0.4151, 0.5849],
        [0.4199, 0.5801],
        [0.4100, 0.5900]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4558, 0.5442],
        [0.4580, 0.5420],
        [0.4634, 0.5366],
        [0.4800, 0.5200],
        [0.4246, 0.5754],
        [0.4611, 0.5389],
        [0.4459, 0.5541],
        [0.4288, 0.5712],
        [0.4419, 0.5581],
        [0.4757, 0.5243],
        [0.4647, 0.5353],
        [0.4401, 0.5599],
        [0.4536, 0.5464],
        [0.4560, 0.5440]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 6 [   0/390]  Loss: 2.220 (2.22)  Acc@1: 48.4375 (48.4375)  Acc@5: 73.4375 (73.4375)LR: 2.416e-02
Train: 6 [  50/390]  Loss: 2.039 (2.14)  Acc@1: 39.0625 (42.8922)  Acc@5: 76.5625 (75.4902)LR: 2.416e-02
Train: 6 [ 100/390]  Loss: 2.070 (2.17)  Acc@1: 43.7500 (42.3886)  Acc@5: 78.1250 (74.5359)LR: 2.416e-02
Train: 6 [ 150/390]  Loss: 2.120 (2.18)  Acc@1: 35.9375 (41.8046)  Acc@5: 76.5625 (74.2860)LR: 2.416e-02
Train: 6 [ 200/390]  Loss: 2.470 (2.19)  Acc@1: 28.1250 (41.4568)  Acc@5: 67.1875 (74.0749)LR: 2.416e-02
Train: 6 [ 250/390]  Loss: 2.000 (2.20)  Acc@1: 45.3125 (41.2911)  Acc@5: 75.0000 (73.9044)LR: 2.416e-02
Train: 6 [ 300/390]  Loss: 2.112 (2.19)  Acc@1: 42.1875 (41.5023)  Acc@5: 73.4375 (73.9151)LR: 2.416e-02
Train: 6 [ 350/390]  Loss: 2.308 (2.18)  Acc@1: 29.6875 (41.7379)  Acc@5: 71.8750 (74.1720)LR: 2.416e-02
Train: 6 [ 390/390]  Loss: 2.419 (2.18)  Acc@1: 42.5000 (41.8720)  Acc@5: 75.0000 (74.2840)LR: 2.416e-02
train_acc 41.872000
Valid: 6 [   0/390]  Loss: 2.580 (2.58)  Acc@1: 32.8125 (32.8125)  Acc@5: 62.5000 (62.5000)
Valid: 6 [  50/390]  Loss: 2.510 (2.26)  Acc@1: 40.6250 (40.7169)  Acc@5: 71.8750 (73.3150)
Valid: 6 [ 100/390]  Loss: 2.180 (2.26)  Acc@1: 45.3125 (40.5941)  Acc@5: 73.4375 (73.0198)
Valid: 6 [ 150/390]  Loss: 2.248 (2.27)  Acc@1: 45.3125 (40.3767)  Acc@5: 68.7500 (72.8891)
Valid: 6 [ 200/390]  Loss: 2.275 (2.27)  Acc@1: 43.7500 (40.5006)  Acc@5: 78.1250 (72.6757)
Valid: 6 [ 250/390]  Loss: 2.094 (2.29)  Acc@1: 46.8750 (40.0710)  Acc@5: 71.8750 (72.5660)
Valid: 6 [ 300/390]  Loss: 2.489 (2.29)  Acc@1: 37.5000 (40.0073)  Acc@5: 70.3125 (72.4927)
Valid: 6 [ 350/390]  Loss: 2.888 (2.29)  Acc@1: 28.1250 (39.9484)  Acc@5: 59.3750 (72.5160)
Valid: 6 [ 390/390]  Loss: 2.060 (2.29)  Acc@1: 45.0000 (39.8840)  Acc@5: 72.5000 (72.5680)
valid_acc 39.884000
epoch = 6   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 1)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4203, 0.5797],
        [0.4095, 0.5905],
        [0.4133, 0.5867],
        [0.4206, 0.5794],
        [0.3941, 0.6059],
        [0.4109, 0.5891],
        [0.4199, 0.5801],
        [0.3899, 0.6101],
        [0.3864, 0.6136],
        [0.4201, 0.5799],
        [0.3942, 0.6058],
        [0.3981, 0.6019],
        [0.3958, 0.6042],
        [0.3902, 0.6098]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4481, 0.5519],
        [0.4484, 0.5516],
        [0.4486, 0.5514],
        [0.4741, 0.5259],
        [0.4154, 0.5846],
        [0.4518, 0.5482],
        [0.4378, 0.5622],
        [0.4129, 0.5871],
        [0.4358, 0.5642],
        [0.4730, 0.5270],
        [0.4560, 0.5440],
        [0.4328, 0.5672],
        [0.4527, 0.5473],
        [0.4518, 0.5482]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 7 [   0/390]  Loss: 2.005 (2.01)  Acc@1: 42.1875 (42.1875)  Acc@5: 78.1250 (78.1250)LR: 2.386e-02
Train: 7 [  50/390]  Loss: 2.391 (2.05)  Acc@1: 29.6875 (43.9338)  Acc@5: 64.0625 (76.6238)LR: 2.386e-02
Train: 7 [ 100/390]  Loss: 1.882 (2.05)  Acc@1: 53.1250 (44.3069)  Acc@5: 78.1250 (76.7946)LR: 2.386e-02
Train: 7 [ 150/390]  Loss: 1.786 (2.05)  Acc@1: 48.4375 (44.3605)  Acc@5: 85.9375 (76.7488)LR: 2.386e-02
Train: 7 [ 200/390]  Loss: 2.099 (2.05)  Acc@1: 39.0625 (44.5662)  Acc@5: 75.0000 (76.5703)LR: 2.386e-02
Train: 7 [ 250/390]  Loss: 2.307 (2.05)  Acc@1: 37.5000 (44.4970)  Acc@5: 70.3125 (76.6310)LR: 2.386e-02
Train: 7 [ 300/390]  Loss: 1.971 (2.06)  Acc@1: 46.8750 (44.3625)  Acc@5: 73.4375 (76.5365)LR: 2.386e-02
Train: 7 [ 350/390]  Loss: 2.044 (2.05)  Acc@1: 50.0000 (44.4444)  Acc@5: 78.1250 (76.6649)LR: 2.386e-02
Train: 7 [ 390/390]  Loss: 2.021 (2.05)  Acc@1: 37.5000 (44.3760)  Acc@5: 82.5000 (76.6200)LR: 2.386e-02
train_acc 44.376000
Valid: 7 [   0/390]  Loss: 2.072 (2.07)  Acc@1: 45.3125 (45.3125)  Acc@5: 78.1250 (78.1250)
Valid: 7 [  50/390]  Loss: 2.137 (2.22)  Acc@1: 43.7500 (41.3297)  Acc@5: 71.8750 (74.1728)
Valid: 7 [ 100/390]  Loss: 2.351 (2.26)  Acc@1: 39.0625 (40.1145)  Acc@5: 71.8750 (73.3137)
Valid: 7 [ 150/390]  Loss: 1.951 (2.26)  Acc@1: 48.4375 (39.8593)  Acc@5: 81.2500 (73.8307)
Valid: 7 [ 200/390]  Loss: 1.723 (2.26)  Acc@1: 53.1250 (40.1430)  Acc@5: 89.0625 (73.9739)
Valid: 7 [ 250/390]  Loss: 2.159 (2.27)  Acc@1: 42.1875 (39.8904)  Acc@5: 76.5625 (73.7737)
Valid: 7 [ 300/390]  Loss: 2.483 (2.27)  Acc@1: 34.3750 (40.0021)  Acc@5: 68.7500 (73.7282)
Valid: 7 [ 350/390]  Loss: 2.281 (2.27)  Acc@1: 45.3125 (40.0107)  Acc@5: 68.7500 (73.6022)
Valid: 7 [ 390/390]  Loss: 2.076 (2.27)  Acc@1: 40.0000 (40.1160)  Acc@5: 80.0000 (73.6120)
valid_acc 40.116000
epoch = 7   
 genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4056, 0.5944],
        [0.3983, 0.6017],
        [0.3943, 0.6057],
        [0.4108, 0.5892],
        [0.3733, 0.6267],
        [0.3932, 0.6068],
        [0.4078, 0.5922],
        [0.3684, 0.6316],
        [0.3645, 0.6355],
        [0.4028, 0.5972],
        [0.3802, 0.6198],
        [0.3794, 0.6206],
        [0.3758, 0.6242],
        [0.3654, 0.6346]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4418, 0.5582],
        [0.4395, 0.5605],
        [0.4358, 0.5642],
        [0.4669, 0.5331],
        [0.3997, 0.6003],
        [0.4433, 0.5567],
        [0.4315, 0.5685],
        [0.4020, 0.5980],
        [0.4300, 0.5700],
        [0.4675, 0.5325],
        [0.4477, 0.5523],
        [0.4186, 0.5814],
        [0.4499, 0.5501],
        [0.4440, 0.5560]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 8 [   0/390]  Loss: 2.204 (2.20)  Acc@1: 35.9375 (35.9375)  Acc@5: 79.6875 (79.6875)LR: 2.352e-02
Train: 8 [  50/390]  Loss: 1.788 (1.87)  Acc@1: 54.6875 (48.4681)  Acc@5: 79.6875 (80.4228)LR: 2.352e-02
Train: 8 [ 100/390]  Loss: 2.200 (1.92)  Acc@1: 37.5000 (47.2308)  Acc@5: 73.4375 (79.1770)LR: 2.352e-02
Train: 8 [ 150/390]  Loss: 2.450 (1.94)  Acc@1: 46.8750 (46.9578)  Acc@5: 71.8750 (79.0149)LR: 2.352e-02
Train: 8 [ 200/390]  Loss: 1.745 (1.95)  Acc@1: 46.8750 (46.4086)  Acc@5: 73.4375 (79.0112)LR: 2.352e-02
Train: 8 [ 250/390]  Loss: 1.964 (1.95)  Acc@1: 50.0000 (46.5139)  Acc@5: 73.4375 (78.8409)LR: 2.352e-02
Train: 8 [ 300/390]  Loss: 2.210 (1.95)  Acc@1: 43.7500 (46.4182)  Acc@5: 73.4375 (78.8621)LR: 2.352e-02
Train: 8 [ 350/390]  Loss: 1.672 (1.95)  Acc@1: 56.2500 (46.6435)  Acc@5: 82.8125 (78.9129)LR: 2.352e-02
Train: 8 [ 390/390]  Loss: 2.354 (1.95)  Acc@1: 37.5000 (46.7120)  Acc@5: 67.5000 (78.8520)LR: 2.352e-02
train_acc 46.712000
Valid: 8 [   0/390]  Loss: 2.251 (2.25)  Acc@1: 42.1875 (42.1875)  Acc@5: 81.2500 (81.2500)
Valid: 8 [  50/390]  Loss: 2.129 (2.07)  Acc@1: 45.3125 (44.9755)  Acc@5: 73.4375 (76.3480)
Valid: 8 [ 100/390]  Loss: 2.281 (2.12)  Acc@1: 35.9375 (43.4870)  Acc@5: 68.7500 (75.5105)
Valid: 8 [ 150/390]  Loss: 2.648 (2.13)  Acc@1: 35.9375 (43.3878)  Acc@5: 65.6250 (75.6519)
Valid: 8 [ 200/390]  Loss: 2.358 (2.13)  Acc@1: 37.5000 (43.4157)  Acc@5: 70.3125 (75.6996)
Valid: 8 [ 250/390]  Loss: 2.102 (2.12)  Acc@1: 42.1875 (43.9305)  Acc@5: 81.2500 (76.1703)
Valid: 8 [ 300/390]  Loss: 2.580 (2.12)  Acc@1: 35.9375 (43.9732)  Acc@5: 68.7500 (76.0382)
Valid: 8 [ 350/390]  Loss: 2.590 (2.12)  Acc@1: 43.7500 (44.1640)  Acc@5: 65.6250 (76.2108)
Valid: 8 [ 390/390]  Loss: 2.198 (2.12)  Acc@1: 32.5000 (44.0680)  Acc@5: 72.5000 (76.2240)
valid_acc 44.068000
epoch = 8   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3849, 0.6151],
        [0.3850, 0.6150],
        [0.3793, 0.6207],
        [0.4036, 0.5964],
        [0.3611, 0.6389],
        [0.3784, 0.6216],
        [0.3986, 0.6014],
        [0.3494, 0.6506],
        [0.3457, 0.6543],
        [0.3870, 0.6130],
        [0.3680, 0.6320],
        [0.3634, 0.6366],
        [0.3556, 0.6444],
        [0.3424, 0.6576]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4283, 0.5717],
        [0.4363, 0.5637],
        [0.4228, 0.5772],
        [0.4609, 0.5391],
        [0.3908, 0.6092],
        [0.4383, 0.5617],
        [0.4251, 0.5749],
        [0.3934, 0.6066],
        [0.4260, 0.5740],
        [0.4623, 0.5377],
        [0.4341, 0.5659],
        [0.4130, 0.5870],
        [0.4481, 0.5519],
        [0.4409, 0.5591]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 9 [   0/390]  Loss: 1.817 (1.82)  Acc@1: 53.1250 (53.1250)  Acc@5: 85.9375 (85.9375)LR: 2.313e-02
Train: 9 [  50/390]  Loss: 1.881 (1.84)  Acc@1: 48.4375 (49.8775)  Acc@5: 79.6875 (80.2696)LR: 2.313e-02
Train: 9 [ 100/390]  Loss: 1.879 (1.84)  Acc@1: 51.5625 (49.5823)  Acc@5: 78.1250 (80.5229)LR: 2.313e-02
Train: 9 [ 150/390]  Loss: 1.644 (1.85)  Acc@1: 56.2500 (49.3481)  Acc@5: 87.5000 (80.5257)LR: 2.313e-02
Train: 9 [ 200/390]  Loss: 1.577 (1.85)  Acc@1: 45.3125 (49.3470)  Acc@5: 87.5000 (80.4415)LR: 2.313e-02
Train: 9 [ 250/390]  Loss: 2.375 (1.86)  Acc@1: 32.8125 (49.2343)  Acc@5: 78.1250 (80.2478)LR: 2.313e-02
Train: 9 [ 300/390]  Loss: 1.684 (1.86)  Acc@1: 50.0000 (49.0708)  Acc@5: 82.8125 (80.3208)LR: 2.313e-02
Train: 9 [ 350/390]  Loss: 1.871 (1.86)  Acc@1: 51.5625 (49.0028)  Acc@5: 68.7500 (80.3908)LR: 2.313e-02
Train: 9 [ 390/390]  Loss: 2.032 (1.87)  Acc@1: 42.5000 (49.0280)  Acc@5: 77.5000 (80.2920)LR: 2.313e-02
train_acc 49.028000
Valid: 9 [   0/390]  Loss: 1.921 (1.92)  Acc@1: 43.7500 (43.7500)  Acc@5: 81.2500 (81.2500)
Valid: 9 [  50/390]  Loss: 1.917 (2.11)  Acc@1: 43.7500 (45.2819)  Acc@5: 82.8125 (76.2561)
Valid: 9 [ 100/390]  Loss: 2.574 (2.11)  Acc@1: 40.6250 (45.2506)  Acc@5: 65.6250 (76.2376)
Valid: 9 [ 150/390]  Loss: 2.019 (2.12)  Acc@1: 43.7500 (44.7434)  Acc@5: 76.5625 (75.6623)
Valid: 9 [ 200/390]  Loss: 2.209 (2.13)  Acc@1: 40.6250 (44.5973)  Acc@5: 73.4375 (75.5053)
Valid: 9 [ 250/390]  Loss: 2.418 (2.14)  Acc@1: 25.0000 (44.2791)  Acc@5: 76.5625 (75.4793)
Valid: 9 [ 300/390]  Loss: 1.664 (2.14)  Acc@1: 51.5625 (44.2172)  Acc@5: 84.3750 (75.4412)
Valid: 9 [ 350/390]  Loss: 2.106 (2.13)  Acc@1: 45.3125 (44.4266)  Acc@5: 76.5625 (75.5030)
Valid: 9 [ 390/390]  Loss: 1.907 (2.13)  Acc@1: 52.5000 (44.6000)  Acc@5: 75.0000 (75.6240)
valid_acc 44.600000
epoch = 9   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3679, 0.6321],
        [0.3760, 0.6240],
        [0.3673, 0.6327],
        [0.3968, 0.6032],
        [0.3445, 0.6555],
        [0.3685, 0.6315],
        [0.3950, 0.6050],
        [0.3298, 0.6702],
        [0.3282, 0.6718],
        [0.3743, 0.6257],
        [0.3596, 0.6404],
        [0.3474, 0.6526],
        [0.3385, 0.6615],
        [0.3243, 0.6757]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4171, 0.5829],
        [0.4316, 0.5684],
        [0.4120, 0.5880],
        [0.4501, 0.5499],
        [0.3831, 0.6169],
        [0.4308, 0.5692],
        [0.4196, 0.5804],
        [0.3840, 0.6160],
        [0.4144, 0.5856],
        [0.4583, 0.5417],
        [0.4211, 0.5789],
        [0.4040, 0.5960],
        [0.4470, 0.5530],
        [0.4339, 0.5661]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 10 [   0/390]  Loss: 1.636 (1.64)  Acc@1: 51.5625 (51.5625)  Acc@5: 82.8125 (82.8125)LR: 2.271e-02
Train: 10 [  50/390]  Loss: 1.553 (1.71)  Acc@1: 54.6875 (52.3897)  Acc@5: 85.9375 (82.5061)LR: 2.271e-02
Train: 10 [ 100/390]  Loss: 1.645 (1.75)  Acc@1: 57.8125 (51.8564)  Acc@5: 82.8125 (82.0390)LR: 2.271e-02
Train: 10 [ 150/390]  Loss: 1.748 (1.76)  Acc@1: 54.6875 (51.6142)  Acc@5: 81.2500 (82.1502)LR: 2.271e-02
Train: 10 [ 200/390]  Loss: 2.018 (1.76)  Acc@1: 40.6250 (51.5625)  Acc@5: 82.8125 (82.3539)LR: 2.271e-02
Train: 10 [ 250/390]  Loss: 1.506 (1.77)  Acc@1: 62.5000 (51.3695)  Acc@5: 90.6250 (82.2647)LR: 2.271e-02
Train: 10 [ 300/390]  Loss: 1.777 (1.76)  Acc@1: 48.4375 (51.3912)  Acc@5: 79.6875 (82.2778)LR: 2.271e-02
Train: 10 [ 350/390]  Loss: 1.848 (1.77)  Acc@1: 46.8750 (51.2331)  Acc@5: 81.2500 (82.2561)LR: 2.271e-02
Train: 10 [ 390/390]  Loss: 2.249 (1.77)  Acc@1: 37.5000 (51.2600)  Acc@5: 72.5000 (82.1880)LR: 2.271e-02
train_acc 51.260000
Valid: 10 [   0/390]  Loss: 1.744 (1.74)  Acc@1: 50.0000 (50.0000)  Acc@5: 79.6875 (79.6875)
Valid: 10 [  50/390]  Loss: 2.128 (2.06)  Acc@1: 43.7500 (45.0674)  Acc@5: 75.0000 (76.8995)
Valid: 10 [ 100/390]  Loss: 1.512 (2.02)  Acc@1: 51.5625 (46.2717)  Acc@5: 85.9375 (77.7382)
Valid: 10 [ 150/390]  Loss: 1.876 (2.03)  Acc@1: 42.1875 (46.0679)  Acc@5: 84.3750 (77.5248)
Valid: 10 [ 200/390]  Loss: 1.637 (2.02)  Acc@1: 46.8750 (46.1132)  Acc@5: 82.8125 (77.8451)
Valid: 10 [ 250/390]  Loss: 1.980 (2.02)  Acc@1: 48.4375 (46.2027)  Acc@5: 81.2500 (77.7888)
Valid: 10 [ 300/390]  Loss: 1.958 (2.03)  Acc@1: 43.7500 (45.8576)  Acc@5: 82.8125 (77.6059)
Valid: 10 [ 350/390]  Loss: 2.096 (2.02)  Acc@1: 46.8750 (46.0114)  Acc@5: 68.7500 (77.7377)
Valid: 10 [ 390/390]  Loss: 2.033 (2.02)  Acc@1: 42.5000 (46.0840)  Acc@5: 77.5000 (77.8280)
valid_acc 46.084000
epoch = 10   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3566, 0.6434],
        [0.3641, 0.6359],
        [0.3536, 0.6464],
        [0.3869, 0.6131],
        [0.3342, 0.6658],
        [0.3604, 0.6396],
        [0.3878, 0.6122],
        [0.3202, 0.6798],
        [0.3137, 0.6863],
        [0.3609, 0.6391],
        [0.3509, 0.6491],
        [0.3340, 0.6660],
        [0.3215, 0.6785],
        [0.3021, 0.6979]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4048, 0.5952],
        [0.4236, 0.5764],
        [0.4006, 0.5994],
        [0.4420, 0.5580],
        [0.3716, 0.6284],
        [0.4232, 0.5768],
        [0.4080, 0.5920],
        [0.3723, 0.6277],
        [0.4074, 0.5926],
        [0.4582, 0.5418],
        [0.4072, 0.5928],
        [0.3945, 0.6055],
        [0.4407, 0.5593],
        [0.4305, 0.5695]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 11 [   0/390]  Loss: 1.456 (1.46)  Acc@1: 59.3750 (59.3750)  Acc@5: 85.9375 (85.9375)LR: 2.225e-02
Train: 11 [  50/390]  Loss: 1.496 (1.65)  Acc@1: 56.2500 (53.2169)  Acc@5: 85.9375 (83.8848)LR: 2.225e-02
Train: 11 [ 100/390]  Loss: 1.760 (1.65)  Acc@1: 53.1250 (53.7593)  Acc@5: 79.6875 (83.7098)LR: 2.225e-02
Train: 11 [ 150/390]  Loss: 1.568 (1.66)  Acc@1: 53.1250 (53.5803)  Acc@5: 89.0625 (83.6300)LR: 2.225e-02
Train: 11 [ 200/390]  Loss: 1.692 (1.67)  Acc@1: 56.2500 (53.2261)  Acc@5: 85.9375 (83.6054)LR: 2.225e-02
Train: 11 [ 250/390]  Loss: 1.641 (1.67)  Acc@1: 54.6875 (53.1810)  Acc@5: 76.5625 (83.5844)LR: 2.225e-02
Train: 11 [ 300/390]  Loss: 1.526 (1.69)  Acc@1: 53.1250 (52.6941)  Acc@5: 79.6875 (83.2693)LR: 2.225e-02
Train: 11 [ 350/390]  Loss: 1.375 (1.69)  Acc@1: 59.3750 (52.5597)  Acc@5: 87.5000 (83.1597)LR: 2.225e-02
Train: 11 [ 390/390]  Loss: 1.090 (1.69)  Acc@1: 62.5000 (52.5880)  Acc@5: 95.0000 (83.1960)LR: 2.225e-02
train_acc 52.588000
Valid: 11 [   0/390]  Loss: 1.635 (1.63)  Acc@1: 59.3750 (59.3750)  Acc@5: 81.2500 (81.2500)
Valid: 11 [  50/390]  Loss: 2.255 (1.98)  Acc@1: 42.1875 (47.9779)  Acc@5: 76.5625 (77.8493)
Valid: 11 [ 100/390]  Loss: 2.368 (2.01)  Acc@1: 42.1875 (47.6949)  Acc@5: 71.8750 (77.4134)
Valid: 11 [ 150/390]  Loss: 2.054 (2.00)  Acc@1: 42.1875 (47.8166)  Acc@5: 79.6875 (78.0008)
Valid: 11 [ 200/390]  Loss: 2.011 (1.99)  Acc@1: 43.7500 (47.6290)  Acc@5: 84.3750 (78.0706)
Valid: 11 [ 250/390]  Loss: 2.554 (2.00)  Acc@1: 34.3750 (47.4041)  Acc@5: 75.0000 (78.0316)
Valid: 11 [ 300/390]  Loss: 1.651 (1.99)  Acc@1: 48.4375 (47.4927)  Acc@5: 84.3750 (78.1925)
Valid: 11 [ 350/390]  Loss: 2.001 (2.00)  Acc@1: 56.2500 (47.5472)  Acc@5: 76.5625 (78.1517)
Valid: 11 [ 390/390]  Loss: 2.318 (2.00)  Acc@1: 37.5000 (47.5120)  Acc@5: 72.5000 (78.1200)
valid_acc 47.512000
epoch = 11   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3433, 0.6567],
        [0.3522, 0.6478],
        [0.3377, 0.6623],
        [0.3776, 0.6224],
        [0.3225, 0.6775],
        [0.3502, 0.6498],
        [0.3816, 0.6184],
        [0.3056, 0.6944],
        [0.2986, 0.7014],
        [0.3475, 0.6525],
        [0.3407, 0.6593],
        [0.3243, 0.6757],
        [0.3064, 0.6936],
        [0.2816, 0.7184]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3912, 0.6088],
        [0.4138, 0.5862],
        [0.3871, 0.6129],
        [0.4318, 0.5682],
        [0.3596, 0.6404],
        [0.4138, 0.5862],
        [0.3953, 0.6047],
        [0.3662, 0.6338],
        [0.3973, 0.6027],
        [0.4592, 0.5408],
        [0.4007, 0.5993],
        [0.3856, 0.6144],
        [0.4355, 0.5645],
        [0.4256, 0.5744]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 12 [   0/390]  Loss: 1.444 (1.44)  Acc@1: 57.8125 (57.8125)  Acc@5: 81.2500 (81.2500)LR: 2.175e-02
Train: 12 [  50/390]  Loss: 1.670 (1.59)  Acc@1: 43.7500 (56.0662)  Acc@5: 84.3750 (85.2941)LR: 2.175e-02
Train: 12 [ 100/390]  Loss: 1.540 (1.61)  Acc@1: 57.8125 (55.5229)  Acc@5: 84.3750 (85.0866)LR: 2.175e-02
Train: 12 [ 150/390]  Loss: 1.362 (1.60)  Acc@1: 59.3750 (54.9565)  Acc@5: 92.1875 (85.1821)LR: 2.175e-02
Train: 12 [ 200/390]  Loss: 1.671 (1.61)  Acc@1: 50.0000 (54.5942)  Acc@5: 85.9375 (84.9891)LR: 2.175e-02
Train: 12 [ 250/390]  Loss: 1.469 (1.60)  Acc@1: 59.3750 (54.7933)  Acc@5: 85.9375 (85.0473)LR: 2.175e-02
Train: 12 [ 300/390]  Loss: 1.294 (1.60)  Acc@1: 64.0625 (54.7394)  Acc@5: 85.9375 (85.0446)LR: 2.175e-02
Train: 12 [ 350/390]  Loss: 1.668 (1.62)  Acc@1: 53.1250 (54.6519)  Acc@5: 82.8125 (84.8558)LR: 2.175e-02
Train: 12 [ 390/390]  Loss: 1.596 (1.62)  Acc@1: 67.5000 (54.5440)  Acc@5: 87.5000 (84.6720)LR: 2.175e-02
train_acc 54.544000
Valid: 12 [   0/390]  Loss: 2.007 (2.01)  Acc@1: 39.0625 (39.0625)  Acc@5: 76.5625 (76.5625)
Valid: 12 [  50/390]  Loss: 1.794 (1.89)  Acc@1: 54.6875 (50.7966)  Acc@5: 81.2500 (79.8713)
Valid: 12 [ 100/390]  Loss: 2.486 (1.92)  Acc@1: 32.8125 (49.5823)  Acc@5: 73.4375 (79.7649)
Valid: 12 [ 150/390]  Loss: 1.981 (1.93)  Acc@1: 43.7500 (49.2757)  Acc@5: 84.3750 (79.0873)
Valid: 12 [ 200/390]  Loss: 2.083 (1.93)  Acc@1: 56.2500 (49.1682)  Acc@5: 71.8750 (79.1511)
Valid: 12 [ 250/390]  Loss: 1.933 (1.94)  Acc@1: 51.5625 (48.8982)  Acc@5: 79.6875 (79.0090)
Valid: 12 [ 300/390]  Loss: 1.870 (1.93)  Acc@1: 45.3125 (48.9099)  Acc@5: 82.8125 (79.1788)
Valid: 12 [ 350/390]  Loss: 2.218 (1.92)  Acc@1: 45.3125 (49.0830)  Acc@5: 68.7500 (79.2023)
Valid: 12 [ 390/390]  Loss: 1.486 (1.92)  Acc@1: 57.5000 (49.0320)  Acc@5: 85.0000 (79.2400)
valid_acc 49.032000
epoch = 12   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3332, 0.6668],
        [0.3432, 0.6568],
        [0.3295, 0.6705],
        [0.3753, 0.6247],
        [0.3117, 0.6883],
        [0.3452, 0.6548],
        [0.3816, 0.6184],
        [0.2928, 0.7072],
        [0.2844, 0.7156],
        [0.3357, 0.6643],
        [0.3339, 0.6661],
        [0.3162, 0.6838],
        [0.2871, 0.7129],
        [0.2609, 0.7391]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3843, 0.6157],
        [0.4101, 0.5899],
        [0.3736, 0.6264],
        [0.4278, 0.5722],
        [0.3541, 0.6459],
        [0.4098, 0.5902],
        [0.3881, 0.6119],
        [0.3571, 0.6429],
        [0.3956, 0.6044],
        [0.4571, 0.5429],
        [0.4010, 0.5990],
        [0.3792, 0.6208],
        [0.4326, 0.5674],
        [0.4247, 0.5753]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 13 [   0/390]  Loss: 1.908 (1.91)  Acc@1: 45.3125 (45.3125)  Acc@5: 79.6875 (79.6875)LR: 2.121e-02
Train: 13 [  50/390]  Loss: 1.381 (1.54)  Acc@1: 60.9375 (57.4755)  Acc@5: 92.1875 (85.9375)LR: 2.121e-02
Train: 13 [ 100/390]  Loss: 1.772 (1.55)  Acc@1: 48.4375 (56.1726)  Acc@5: 85.9375 (86.0149)LR: 2.121e-02
Train: 13 [ 150/390]  Loss: 1.700 (1.54)  Acc@1: 51.5625 (56.2707)  Acc@5: 82.8125 (85.9996)LR: 2.121e-02
Train: 13 [ 200/390]  Loss: 1.681 (1.56)  Acc@1: 51.5625 (55.9701)  Acc@5: 84.3750 (85.5721)LR: 2.121e-02
Train: 13 [ 250/390]  Loss: 1.674 (1.56)  Acc@1: 51.5625 (56.0383)  Acc@5: 78.1250 (85.4893)LR: 2.121e-02
Train: 13 [ 300/390]  Loss: 1.833 (1.56)  Acc@1: 48.4375 (55.9022)  Acc@5: 79.6875 (85.4651)LR: 2.121e-02
Train: 13 [ 350/390]  Loss: 1.932 (1.56)  Acc@1: 43.7500 (55.7959)  Acc@5: 84.3750 (85.5191)LR: 2.121e-02
Train: 13 [ 390/390]  Loss: 1.618 (1.57)  Acc@1: 45.0000 (55.6640)  Acc@5: 90.0000 (85.4320)LR: 2.121e-02
train_acc 55.664000
Valid: 13 [   0/390]  Loss: 1.855 (1.85)  Acc@1: 54.6875 (54.6875)  Acc@5: 79.6875 (79.6875)
Valid: 13 [  50/390]  Loss: 1.714 (1.92)  Acc@1: 57.8125 (49.6324)  Acc@5: 79.6875 (79.7794)
Valid: 13 [ 100/390]  Loss: 2.332 (1.89)  Acc@1: 40.6250 (50.0464)  Acc@5: 67.1875 (80.4765)
Valid: 13 [ 150/390]  Loss: 2.026 (1.87)  Acc@1: 45.3125 (50.4450)  Acc@5: 85.9375 (80.5153)
Valid: 13 [ 200/390]  Loss: 1.901 (1.87)  Acc@1: 51.5625 (50.5519)  Acc@5: 85.9375 (80.6748)
Valid: 13 [ 250/390]  Loss: 2.004 (1.88)  Acc@1: 46.8750 (50.2428)  Acc@5: 78.1250 (80.5839)
Valid: 13 [ 300/390]  Loss: 2.139 (1.89)  Acc@1: 46.8750 (49.6418)  Acc@5: 71.8750 (80.2585)
Valid: 13 [ 350/390]  Loss: 1.855 (1.90)  Acc@1: 50.0000 (49.6394)  Acc@5: 75.0000 (80.1371)
Valid: 13 [ 390/390]  Loss: 2.387 (1.89)  Acc@1: 45.0000 (49.7200)  Acc@5: 70.0000 (80.1360)
valid_acc 49.720000
epoch = 13   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3235, 0.6765],
        [0.3358, 0.6642],
        [0.3217, 0.6783],
        [0.3673, 0.6327],
        [0.3032, 0.6968],
        [0.3385, 0.6615],
        [0.3789, 0.6211],
        [0.2827, 0.7173],
        [0.2723, 0.7277],
        [0.3271, 0.6729],
        [0.3276, 0.6724],
        [0.3066, 0.6934],
        [0.2713, 0.7287],
        [0.2435, 0.7565]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3781, 0.6219],
        [0.4019, 0.5981],
        [0.3624, 0.6376],
        [0.4192, 0.5808],
        [0.3450, 0.6550],
        [0.4081, 0.5919],
        [0.3768, 0.6232],
        [0.3536, 0.6464],
        [0.3922, 0.6078],
        [0.4567, 0.5433],
        [0.3909, 0.6091],
        [0.3701, 0.6299],
        [0.4310, 0.5690],
        [0.4187, 0.5813]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 14 [   0/390]  Loss: 1.549 (1.55)  Acc@1: 54.6875 (54.6875)  Acc@5: 89.0625 (89.0625)LR: 2.065e-02
Train: 14 [  50/390]  Loss: 1.540 (1.43)  Acc@1: 50.0000 (58.7929)  Acc@5: 87.5000 (87.1936)LR: 2.065e-02
Train: 14 [ 100/390]  Loss: 1.162 (1.45)  Acc@1: 64.0625 (58.1993)  Acc@5: 93.7500 (87.0668)LR: 2.065e-02
Train: 14 [ 150/390]  Loss: 1.446 (1.47)  Acc@1: 62.5000 (58.0298)  Acc@5: 87.5000 (86.9205)LR: 2.065e-02
Train: 14 [ 200/390]  Loss: 1.890 (1.48)  Acc@1: 48.4375 (57.7581)  Acc@5: 81.2500 (86.8004)LR: 2.065e-02
Train: 14 [ 250/390]  Loss: 1.478 (1.48)  Acc@1: 54.6875 (57.8436)  Acc@5: 87.5000 (86.7530)LR: 2.065e-02
Train: 14 [ 300/390]  Loss: 1.495 (1.49)  Acc@1: 57.8125 (57.4647)  Acc@5: 89.0625 (86.4255)LR: 2.065e-02
Train: 14 [ 350/390]  Loss: 1.519 (1.50)  Acc@1: 56.2500 (57.4964)  Acc@5: 82.8125 (86.3827)LR: 2.065e-02
Train: 14 [ 390/390]  Loss: 1.485 (1.50)  Acc@1: 57.5000 (57.4720)  Acc@5: 87.5000 (86.3360)LR: 2.065e-02
train_acc 57.472000
Valid: 14 [   0/390]  Loss: 1.979 (1.98)  Acc@1: 46.8750 (46.8750)  Acc@5: 78.1250 (78.1250)
Valid: 14 [  50/390]  Loss: 2.013 (1.82)  Acc@1: 42.1875 (51.5012)  Acc@5: 82.8125 (81.6176)
Valid: 14 [ 100/390]  Loss: 1.985 (1.82)  Acc@1: 50.0000 (51.6244)  Acc@5: 78.1250 (80.9097)
Valid: 14 [ 150/390]  Loss: 1.088 (1.81)  Acc@1: 64.0625 (51.2728)  Acc@5: 96.8750 (81.0637)
Valid: 14 [ 200/390]  Loss: 1.687 (1.82)  Acc@1: 51.5625 (51.0961)  Acc@5: 84.3750 (81.3511)
Valid: 14 [ 250/390]  Loss: 2.371 (1.83)  Acc@1: 42.1875 (50.9151)  Acc@5: 73.4375 (81.2002)
Valid: 14 [ 300/390]  Loss: 1.722 (1.82)  Acc@1: 43.7500 (51.0590)  Acc@5: 87.5000 (81.2915)
Valid: 14 [ 350/390]  Loss: 1.479 (1.83)  Acc@1: 65.6250 (50.9304)  Acc@5: 87.5000 (81.1120)
Valid: 14 [ 390/390]  Loss: 2.024 (1.83)  Acc@1: 47.5000 (50.9520)  Acc@5: 82.5000 (81.1320)
valid_acc 50.952000
epoch = 14   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3139, 0.6861],
        [0.3272, 0.6728],
        [0.3128, 0.6872],
        [0.3625, 0.6375],
        [0.2954, 0.7046],
        [0.3335, 0.6665],
        [0.3790, 0.6210],
        [0.2770, 0.7230],
        [0.2596, 0.7404],
        [0.3195, 0.6805],
        [0.3244, 0.6756],
        [0.3039, 0.6961],
        [0.2566, 0.7434],
        [0.2258, 0.7742]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3723, 0.6277],
        [0.3952, 0.6048],
        [0.3532, 0.6468],
        [0.4148, 0.5852],
        [0.3397, 0.6603],
        [0.4049, 0.5951],
        [0.3651, 0.6349],
        [0.3491, 0.6509],
        [0.3840, 0.6160],
        [0.4553, 0.5447],
        [0.3823, 0.6177],
        [0.3691, 0.6309],
        [0.4307, 0.5693],
        [0.4146, 0.5854]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 15 [   0/390]  Loss: 1.553 (1.55)  Acc@1: 51.5625 (51.5625)  Acc@5: 84.3750 (84.3750)LR: 2.005e-02
Train: 15 [  50/390]  Loss: 1.500 (1.37)  Acc@1: 60.9375 (60.4167)  Acc@5: 82.8125 (88.2966)LR: 2.005e-02
Train: 15 [ 100/390]  Loss: 1.487 (1.38)  Acc@1: 62.5000 (60.7054)  Acc@5: 87.5000 (88.1498)LR: 2.005e-02
Train: 15 [ 150/390]  Loss: 1.371 (1.39)  Acc@1: 57.8125 (60.5546)  Acc@5: 93.7500 (88.1623)LR: 2.005e-02
Train: 15 [ 200/390]  Loss: 1.460 (1.41)  Acc@1: 56.2500 (59.8881)  Acc@5: 89.0625 (87.9042)LR: 2.005e-02
Train: 15 [ 250/390]  Loss: 1.770 (1.43)  Acc@1: 51.5625 (59.4684)  Acc@5: 84.3750 (87.6619)LR: 2.005e-02
Train: 15 [ 300/390]  Loss: 1.441 (1.44)  Acc@1: 60.9375 (59.3283)  Acc@5: 84.3750 (87.4792)LR: 2.005e-02
Train: 15 [ 350/390]  Loss: 1.689 (1.44)  Acc@1: 45.3125 (59.1480)  Acc@5: 82.8125 (87.3754)LR: 2.005e-02
Train: 15 [ 390/390]  Loss: 1.410 (1.44)  Acc@1: 57.5000 (59.2400)  Acc@5: 85.0000 (87.4440)LR: 2.005e-02
train_acc 59.240000
Valid: 15 [   0/390]  Loss: 1.885 (1.88)  Acc@1: 48.4375 (48.4375)  Acc@5: 82.8125 (82.8125)
Valid: 15 [  50/390]  Loss: 2.004 (1.86)  Acc@1: 46.8750 (50.5208)  Acc@5: 81.2500 (81.5257)
Valid: 15 [ 100/390]  Loss: 1.736 (1.87)  Acc@1: 56.2500 (50.9592)  Acc@5: 81.2500 (81.2191)
Valid: 15 [ 150/390]  Loss: 1.947 (1.89)  Acc@1: 48.4375 (50.4760)  Acc@5: 81.2500 (81.0844)
Valid: 15 [ 200/390]  Loss: 2.103 (1.89)  Acc@1: 45.3125 (50.5752)  Acc@5: 75.0000 (81.1800)
Valid: 15 [ 250/390]  Loss: 1.414 (1.88)  Acc@1: 67.1875 (50.4669)  Acc@5: 85.9375 (81.0072)
Valid: 15 [ 300/390]  Loss: 2.046 (1.88)  Acc@1: 50.0000 (50.4101)  Acc@5: 82.8125 (81.0216)
Valid: 15 [ 350/390]  Loss: 1.736 (1.89)  Acc@1: 56.2500 (50.2760)  Acc@5: 85.9375 (80.9651)
Valid: 15 [ 390/390]  Loss: 2.119 (1.88)  Acc@1: 45.0000 (50.4880)  Acc@5: 75.0000 (81.0600)
valid_acc 50.488000
epoch = 15   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3036, 0.6964],
        [0.3215, 0.6785],
        [0.3054, 0.6946],
        [0.3616, 0.6384],
        [0.2831, 0.7169],
        [0.3296, 0.6704],
        [0.3815, 0.6185],
        [0.2677, 0.7323],
        [0.2480, 0.7520],
        [0.3101, 0.6899],
        [0.3255, 0.6745],
        [0.2941, 0.7059],
        [0.2418, 0.7582],
        [0.2074, 0.7926]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3649, 0.6351],
        [0.3915, 0.6085],
        [0.3450, 0.6550],
        [0.4065, 0.5935],
        [0.3316, 0.6684],
        [0.4019, 0.5981],
        [0.3557, 0.6443],
        [0.3396, 0.6604],
        [0.3777, 0.6223],
        [0.4522, 0.5478],
        [0.3783, 0.6217],
        [0.3624, 0.6376],
        [0.4282, 0.5718],
        [0.4087, 0.5913]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 16 [   0/390]  Loss: 1.075 (1.08)  Acc@1: 70.3125 (70.3125)  Acc@5: 92.1875 (92.1875)LR: 1.943e-02
Train: 16 [  50/390]  Loss: 1.187 (1.31)  Acc@1: 65.6250 (62.6225)  Acc@5: 92.1875 (89.5833)LR: 1.943e-02
Train: 16 [ 100/390]  Loss: 1.650 (1.33)  Acc@1: 53.1250 (61.8038)  Acc@5: 85.9375 (89.4183)LR: 1.943e-02
Train: 16 [ 150/390]  Loss: 1.292 (1.35)  Acc@1: 64.0625 (61.3618)  Acc@5: 89.0625 (88.8762)LR: 1.943e-02
Train: 16 [ 200/390]  Loss: 1.057 (1.36)  Acc@1: 65.6250 (60.6654)  Acc@5: 96.8750 (88.7438)LR: 1.943e-02
Train: 16 [ 250/390]  Loss: 1.278 (1.38)  Acc@1: 68.7500 (60.4457)  Acc@5: 87.5000 (88.5520)LR: 1.943e-02
Train: 16 [ 300/390]  Loss: 1.537 (1.38)  Acc@1: 59.3750 (60.1381)  Acc@5: 82.8125 (88.4084)LR: 1.943e-02
Train: 16 [ 350/390]  Loss: 1.450 (1.40)  Acc@1: 51.5625 (59.8513)  Acc@5: 89.0625 (88.1499)LR: 1.943e-02
Train: 16 [ 390/390]  Loss: 1.350 (1.40)  Acc@1: 67.5000 (59.7360)  Acc@5: 95.0000 (88.0960)LR: 1.943e-02
train_acc 59.736000
Valid: 16 [   0/390]  Loss: 1.928 (1.93)  Acc@1: 54.6875 (54.6875)  Acc@5: 76.5625 (76.5625)
Valid: 16 [  50/390]  Loss: 1.776 (1.77)  Acc@1: 53.1250 (52.6961)  Acc@5: 78.1250 (81.7708)
Valid: 16 [ 100/390]  Loss: 2.334 (1.76)  Acc@1: 48.4375 (53.1714)  Acc@5: 75.0000 (81.9462)
Valid: 16 [ 150/390]  Loss: 1.875 (1.77)  Acc@1: 46.8750 (53.1457)  Acc@5: 79.6875 (81.7674)
Valid: 16 [ 200/390]  Loss: 2.214 (1.77)  Acc@1: 46.8750 (53.2960)  Acc@5: 78.1250 (81.8797)
Valid: 16 [ 250/390]  Loss: 1.534 (1.78)  Acc@1: 64.0625 (53.2371)  Acc@5: 82.8125 (81.8040)
Valid: 16 [ 300/390]  Loss: 2.155 (1.78)  Acc@1: 43.7500 (53.1561)  Acc@5: 76.5625 (81.7535)
Valid: 16 [ 350/390]  Loss: 2.012 (1.78)  Acc@1: 50.0000 (53.0493)  Acc@5: 75.0000 (81.7441)
Valid: 16 [ 390/390]  Loss: 1.705 (1.78)  Acc@1: 52.5000 (52.9040)  Acc@5: 80.0000 (81.6680)
valid_acc 52.904000
epoch = 16   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2906, 0.7094],
        [0.3148, 0.6852],
        [0.2971, 0.7029],
        [0.3642, 0.6358],
        [0.2772, 0.7228],
        [0.3260, 0.6740],
        [0.3833, 0.6167],
        [0.2617, 0.7383],
        [0.2357, 0.7643],
        [0.3029, 0.6971],
        [0.3269, 0.6731],
        [0.2885, 0.7115],
        [0.2255, 0.7745],
        [0.1913, 0.8087]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3613, 0.6387],
        [0.3841, 0.6159],
        [0.3337, 0.6663],
        [0.3995, 0.6005],
        [0.3205, 0.6795],
        [0.3974, 0.6026],
        [0.3446, 0.6554],
        [0.3272, 0.6728],
        [0.3723, 0.6277],
        [0.4481, 0.5519],
        [0.3746, 0.6254],
        [0.3543, 0.6457],
        [0.4274, 0.5726],
        [0.4013, 0.5987]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 17 [   0/390]  Loss: 1.082 (1.08)  Acc@1: 67.1875 (67.1875)  Acc@5: 87.5000 (87.5000)LR: 1.878e-02
Train: 17 [  50/390]  Loss: 1.290 (1.30)  Acc@1: 67.1875 (63.5723)  Acc@5: 84.3750 (88.8174)LR: 1.878e-02
Train: 17 [ 100/390]  Loss: 1.237 (1.32)  Acc@1: 64.0625 (62.6702)  Acc@5: 90.6250 (88.8769)LR: 1.878e-02
Train: 17 [ 150/390]  Loss: 1.517 (1.32)  Acc@1: 51.5625 (62.6552)  Acc@5: 82.8125 (88.9487)LR: 1.878e-02
Train: 17 [ 200/390]  Loss: 1.422 (1.33)  Acc@1: 56.2500 (62.1891)  Acc@5: 89.0625 (88.9692)LR: 1.878e-02
Train: 17 [ 250/390]  Loss: 1.416 (1.33)  Acc@1: 53.1250 (61.9895)  Acc@5: 89.0625 (89.0127)LR: 1.878e-02
Train: 17 [ 300/390]  Loss: 1.395 (1.34)  Acc@1: 65.6250 (61.9082)  Acc@5: 85.9375 (88.8081)LR: 1.878e-02
Train: 17 [ 350/390]  Loss: 1.425 (1.34)  Acc@1: 59.3750 (61.8412)  Acc@5: 90.6250 (88.9379)LR: 1.878e-02
Train: 17 [ 390/390]  Loss: 1.701 (1.33)  Acc@1: 45.0000 (61.7560)  Acc@5: 85.0000 (89.0200)LR: 1.878e-02
train_acc 61.756000
Valid: 17 [   0/390]  Loss: 1.816 (1.82)  Acc@1: 57.8125 (57.8125)  Acc@5: 90.6250 (90.6250)
Valid: 17 [  50/390]  Loss: 1.579 (1.84)  Acc@1: 59.3750 (52.1140)  Acc@5: 82.8125 (81.7096)
Valid: 17 [ 100/390]  Loss: 1.892 (1.88)  Acc@1: 46.8750 (51.1293)  Acc@5: 78.1250 (81.4821)
Valid: 17 [ 150/390]  Loss: 1.866 (1.87)  Acc@1: 50.0000 (51.4590)  Acc@5: 85.9375 (81.3638)
Valid: 17 [ 200/390]  Loss: 1.535 (1.86)  Acc@1: 54.6875 (51.6713)  Acc@5: 81.2500 (81.5532)
Valid: 17 [ 250/390]  Loss: 2.253 (1.85)  Acc@1: 45.3125 (51.9298)  Acc@5: 76.5625 (81.5986)
Valid: 17 [ 300/390]  Loss: 2.100 (1.85)  Acc@1: 43.7500 (51.8895)  Acc@5: 78.1250 (81.6341)
Valid: 17 [ 350/390]  Loss: 1.648 (1.84)  Acc@1: 59.3750 (51.9943)  Acc@5: 81.2500 (81.8064)
Valid: 17 [ 390/390]  Loss: 2.092 (1.83)  Acc@1: 45.0000 (52.0200)  Acc@5: 77.5000 (81.8240)
valid_acc 52.020000
epoch = 17   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2789, 0.7211],
        [0.3073, 0.6927],
        [0.2911, 0.7089],
        [0.3687, 0.6313],
        [0.2722, 0.7278],
        [0.3247, 0.6753],
        [0.3847, 0.6153],
        [0.2561, 0.7439],
        [0.2231, 0.7769],
        [0.2974, 0.7026],
        [0.3238, 0.6762],
        [0.2828, 0.7172],
        [0.2132, 0.7868],
        [0.1750, 0.8250]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3570, 0.6430],
        [0.3763, 0.6237],
        [0.3271, 0.6729],
        [0.3883, 0.6117],
        [0.3155, 0.6845],
        [0.3976, 0.6024],
        [0.3337, 0.6663],
        [0.3206, 0.6794],
        [0.3640, 0.6360],
        [0.4419, 0.5581],
        [0.3712, 0.6288],
        [0.3478, 0.6522],
        [0.4247, 0.5753],
        [0.3977, 0.6023]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 18 [   0/390]  Loss: 1.397 (1.40)  Acc@1: 59.3750 (59.3750)  Acc@5: 89.0625 (89.0625)LR: 1.811e-02
Train: 18 [  50/390]  Loss: 0.9837 (1.22)  Acc@1: 68.7500 (64.9816)  Acc@5: 93.7500 (90.1961)LR: 1.811e-02
Train: 18 [ 100/390]  Loss: 1.368 (1.23)  Acc@1: 57.8125 (64.4183)  Acc@5: 90.6250 (90.2537)LR: 1.811e-02
Train: 18 [ 150/390]  Loss: 1.276 (1.24)  Acc@1: 51.5625 (63.9590)  Acc@5: 90.6250 (90.2007)LR: 1.811e-02
Train: 18 [ 200/390]  Loss: 1.088 (1.25)  Acc@1: 65.6250 (63.4873)  Acc@5: 90.6250 (90.1897)LR: 1.811e-02
Train: 18 [ 250/390]  Loss: 1.458 (1.27)  Acc@1: 57.8125 (63.1848)  Acc@5: 87.5000 (89.9963)LR: 1.811e-02
Train: 18 [ 300/390]  Loss: 1.714 (1.28)  Acc@1: 51.5625 (62.9464)  Acc@5: 87.5000 (89.8827)LR: 1.811e-02
Train: 18 [ 350/390]  Loss: 1.345 (1.29)  Acc@1: 59.3750 (62.7448)  Acc@5: 90.6250 (89.7837)LR: 1.811e-02
Train: 18 [ 390/390]  Loss: 1.258 (1.29)  Acc@1: 62.5000 (62.6160)  Acc@5: 87.5000 (89.7400)LR: 1.811e-02
train_acc 62.616000
Valid: 18 [   0/390]  Loss: 1.930 (1.93)  Acc@1: 46.8750 (46.8750)  Acc@5: 85.9375 (85.9375)
Valid: 18 [  50/390]  Loss: 1.667 (1.78)  Acc@1: 53.1250 (53.2169)  Acc@5: 85.9375 (82.6287)
Valid: 18 [ 100/390]  Loss: 1.820 (1.78)  Acc@1: 48.4375 (52.7692)  Acc@5: 79.6875 (82.5495)
Valid: 18 [ 150/390]  Loss: 1.428 (1.78)  Acc@1: 57.8125 (52.8249)  Acc@5: 82.8125 (82.5538)
Valid: 18 [ 200/390]  Loss: 2.024 (1.79)  Acc@1: 43.7500 (52.7363)  Acc@5: 82.8125 (82.4083)
Valid: 18 [ 250/390]  Loss: 1.940 (1.78)  Acc@1: 48.4375 (52.8075)  Acc@5: 81.2500 (82.5199)
Valid: 18 [ 300/390]  Loss: 2.000 (1.78)  Acc@1: 50.0000 (52.7461)  Acc@5: 78.1250 (82.5997)
Valid: 18 [ 350/390]  Loss: 2.074 (1.77)  Acc@1: 51.5625 (53.0048)  Acc@5: 75.0000 (82.7502)
Valid: 18 [ 390/390]  Loss: 1.286 (1.77)  Acc@1: 67.5000 (53.2800)  Acc@5: 92.5000 (82.7640)
valid_acc 53.280000
epoch = 18   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2729, 0.7271],
        [0.2980, 0.7020],
        [0.2841, 0.7159],
        [0.3690, 0.6310],
        [0.2684, 0.7316],
        [0.3260, 0.6740],
        [0.3909, 0.6091],
        [0.2568, 0.7432],
        [0.2131, 0.7869],
        [0.2957, 0.7043],
        [0.3304, 0.6696],
        [0.2860, 0.7140],
        [0.2023, 0.7977],
        [0.1628, 0.8372]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3517, 0.6483],
        [0.3635, 0.6365],
        [0.3180, 0.6820],
        [0.3832, 0.6168],
        [0.3115, 0.6885],
        [0.3947, 0.6053],
        [0.3233, 0.6767],
        [0.3147, 0.6853],
        [0.3594, 0.6406],
        [0.4395, 0.5605],
        [0.3695, 0.6305],
        [0.3420, 0.6580],
        [0.4165, 0.5835],
        [0.3913, 0.6087]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 19 [   0/390]  Loss: 1.208 (1.21)  Acc@1: 62.5000 (62.5000)  Acc@5: 90.6250 (90.6250)LR: 1.742e-02
Train: 19 [  50/390]  Loss: 1.184 (1.15)  Acc@1: 65.6250 (66.4522)  Acc@5: 90.6250 (91.3603)LR: 1.742e-02
Train: 19 [ 100/390]  Loss: 1.359 (1.19)  Acc@1: 62.5000 (65.7178)  Acc@5: 90.6250 (90.8725)LR: 1.742e-02
Train: 19 [ 150/390]  Loss: 1.359 (1.20)  Acc@1: 68.7500 (65.5629)  Acc@5: 87.5000 (90.8320)LR: 1.742e-02
Train: 19 [ 200/390]  Loss: 0.9994 (1.21)  Acc@1: 68.7500 (65.0731)  Acc@5: 93.7500 (90.8116)LR: 1.742e-02
Train: 19 [ 250/390]  Loss: 1.505 (1.22)  Acc@1: 60.9375 (64.5605)  Acc@5: 84.3750 (90.6873)LR: 1.742e-02
Train: 19 [ 300/390]  Loss: 1.317 (1.22)  Acc@1: 64.0625 (64.5712)  Acc@5: 92.1875 (90.6042)LR: 1.742e-02
Train: 19 [ 350/390]  Loss: 1.270 (1.23)  Acc@1: 62.5000 (64.3563)  Acc@5: 90.6250 (90.5093)LR: 1.742e-02
Train: 19 [ 390/390]  Loss: 1.220 (1.24)  Acc@1: 62.5000 (64.2280)  Acc@5: 87.5000 (90.4000)LR: 1.742e-02
train_acc 64.228000
Valid: 19 [   0/390]  Loss: 1.853 (1.85)  Acc@1: 43.7500 (43.7500)  Acc@5: 82.8125 (82.8125)
Valid: 19 [  50/390]  Loss: 1.918 (1.84)  Acc@1: 50.0000 (52.5123)  Acc@5: 79.6875 (81.4951)
Valid: 19 [ 100/390]  Loss: 2.137 (1.78)  Acc@1: 42.1875 (53.8676)  Acc@5: 78.1250 (82.2710)
Valid: 19 [ 150/390]  Loss: 1.930 (1.79)  Acc@1: 53.1250 (53.8183)  Acc@5: 84.3750 (82.2434)
Valid: 19 [ 200/390]  Loss: 1.610 (1.78)  Acc@1: 59.3750 (53.7780)  Acc@5: 90.6250 (82.5637)
Valid: 19 [ 250/390]  Loss: 1.812 (1.77)  Acc@1: 57.8125 (53.9280)  Acc@5: 84.3750 (82.7129)
Valid: 19 [ 300/390]  Loss: 1.578 (1.76)  Acc@1: 56.2500 (53.9348)  Acc@5: 85.9375 (82.9630)
Valid: 19 [ 350/390]  Loss: 1.719 (1.76)  Acc@1: 57.8125 (53.9085)  Acc@5: 81.2500 (82.9639)
Valid: 19 [ 390/390]  Loss: 1.521 (1.76)  Acc@1: 50.0000 (53.8560)  Acc@5: 90.0000 (82.9920)
valid_acc 53.856000
epoch = 19   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2669, 0.7331],
        [0.2910, 0.7090],
        [0.2760, 0.7240],
        [0.3687, 0.6313],
        [0.2627, 0.7373],
        [0.3239, 0.6761],
        [0.3951, 0.6049],
        [0.2519, 0.7481],
        [0.2034, 0.7966],
        [0.2923, 0.7077],
        [0.3326, 0.6674],
        [0.2861, 0.7139],
        [0.1896, 0.8104],
        [0.1513, 0.8487]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3463, 0.6537],
        [0.3532, 0.6468],
        [0.3105, 0.6895],
        [0.3754, 0.6246],
        [0.3067, 0.6933],
        [0.3918, 0.6082],
        [0.3139, 0.6861],
        [0.3101, 0.6899],
        [0.3540, 0.6460],
        [0.4400, 0.5600],
        [0.3639, 0.6361],
        [0.3344, 0.6656],
        [0.4132, 0.5868],
        [0.3842, 0.6158]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 20 [   0/390]  Loss: 1.482 (1.48)  Acc@1: 56.2500 (56.2500)  Acc@5: 87.5000 (87.5000)LR: 1.671e-02
Train: 20 [  50/390]  Loss: 1.216 (1.14)  Acc@1: 67.1875 (66.7586)  Acc@5: 87.5000 (92.2181)LR: 1.671e-02
Train: 20 [ 100/390]  Loss: 1.278 (1.14)  Acc@1: 62.5000 (67.0637)  Acc@5: 89.0625 (92.0792)LR: 1.671e-02
Train: 20 [ 150/390]  Loss: 1.318 (1.16)  Acc@1: 51.5625 (66.6701)  Acc@5: 95.3125 (91.5666)LR: 1.671e-02
Train: 20 [ 200/390]  Loss: 1.175 (1.16)  Acc@1: 68.7500 (66.6356)  Acc@5: 90.6250 (91.4956)LR: 1.671e-02
Train: 20 [ 250/390]  Loss: 0.9771 (1.18)  Acc@1: 70.3125 (66.2102)  Acc@5: 93.7500 (91.2039)LR: 1.671e-02
Train: 20 [ 300/390]  Loss: 1.215 (1.18)  Acc@1: 60.9375 (65.9001)  Acc@5: 90.6250 (91.1441)LR: 1.671e-02
Train: 20 [ 350/390]  Loss: 1.030 (1.19)  Acc@1: 70.3125 (65.6695)  Acc@5: 92.1875 (91.0880)LR: 1.671e-02
Train: 20 [ 390/390]  Loss: 1.751 (1.19)  Acc@1: 60.0000 (65.4920)  Acc@5: 82.5000 (91.0160)LR: 1.671e-02
train_acc 65.492000
Valid: 20 [   0/390]  Loss: 1.880 (1.88)  Acc@1: 56.2500 (56.2500)  Acc@5: 79.6875 (79.6875)
Valid: 20 [  50/390]  Loss: 1.377 (1.80)  Acc@1: 54.6875 (54.3811)  Acc@5: 89.0625 (82.4142)
Valid: 20 [ 100/390]  Loss: 1.897 (1.77)  Acc@1: 57.8125 (54.1306)  Acc@5: 76.5625 (82.8434)
Valid: 20 [ 150/390]  Loss: 2.012 (1.77)  Acc@1: 48.4375 (53.9735)  Acc@5: 81.2500 (82.8332)
Valid: 20 [ 200/390]  Loss: 1.455 (1.76)  Acc@1: 60.9375 (53.7780)  Acc@5: 89.0625 (83.1157)
Valid: 20 [ 250/390]  Loss: 1.883 (1.77)  Acc@1: 56.2500 (53.5483)  Acc@5: 78.1250 (82.9681)
Valid: 20 [ 300/390]  Loss: 1.324 (1.76)  Acc@1: 60.9375 (53.5610)  Acc@5: 89.0625 (82.9734)
Valid: 20 [ 350/390]  Loss: 1.746 (1.77)  Acc@1: 54.6875 (53.5345)  Acc@5: 82.8125 (82.9193)
Valid: 20 [ 390/390]  Loss: 2.052 (1.78)  Acc@1: 37.5000 (53.4280)  Acc@5: 85.0000 (82.7720)
valid_acc 53.428000
epoch = 20   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2560, 0.7440],
        [0.2848, 0.7152],
        [0.2689, 0.7311],
        [0.3677, 0.6323],
        [0.2582, 0.7418],
        [0.3195, 0.6805],
        [0.3973, 0.6027],
        [0.2469, 0.7531],
        [0.1927, 0.8073],
        [0.2904, 0.7096],
        [0.3345, 0.6655],
        [0.2837, 0.7163],
        [0.1772, 0.8228],
        [0.1395, 0.8605]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3434, 0.6566],
        [0.3429, 0.6571],
        [0.3030, 0.6970],
        [0.3701, 0.6299],
        [0.2997, 0.7003],
        [0.3879, 0.6121],
        [0.3026, 0.6974],
        [0.3004, 0.6996],
        [0.3476, 0.6524],
        [0.4349, 0.5651],
        [0.3568, 0.6432],
        [0.3252, 0.6748],
        [0.4108, 0.5892],
        [0.3797, 0.6203]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 21 [   0/390]  Loss: 1.138 (1.14)  Acc@1: 68.7500 (68.7500)  Acc@5: 90.6250 (90.6250)LR: 1.598e-02
Train: 21 [  50/390]  Loss: 1.301 (1.09)  Acc@1: 65.6250 (67.6777)  Acc@5: 89.0625 (92.2794)LR: 1.598e-02
Train: 21 [ 100/390]  Loss: 1.050 (1.09)  Acc@1: 67.1875 (67.7444)  Acc@5: 92.1875 (92.5124)LR: 1.598e-02
Train: 21 [ 150/390]  Loss: 1.273 (1.10)  Acc@1: 67.1875 (67.4358)  Acc@5: 93.7500 (92.1565)LR: 1.598e-02
Train: 21 [ 200/390]  Loss: 1.049 (1.11)  Acc@1: 62.5000 (67.3197)  Acc@5: 93.7500 (91.8999)LR: 1.598e-02
Train: 21 [ 250/390]  Loss: 1.147 (1.12)  Acc@1: 67.1875 (67.0630)  Acc@5: 95.3125 (91.9260)LR: 1.598e-02
Train: 21 [ 300/390]  Loss: 1.473 (1.13)  Acc@1: 53.1250 (66.7255)  Acc@5: 89.0625 (91.7307)LR: 1.598e-02
Train: 21 [ 350/390]  Loss: 1.268 (1.14)  Acc@1: 59.3750 (66.3150)  Acc@5: 87.5000 (91.5420)LR: 1.598e-02
Train: 21 [ 390/390]  Loss: 1.212 (1.15)  Acc@1: 62.5000 (66.1880)  Acc@5: 92.5000 (91.5680)LR: 1.598e-02
train_acc 66.188000
Valid: 21 [   0/390]  Loss: 1.670 (1.67)  Acc@1: 62.5000 (62.5000)  Acc@5: 85.9375 (85.9375)
Valid: 21 [  50/390]  Loss: 1.814 (1.73)  Acc@1: 62.5000 (55.7904)  Acc@5: 84.3750 (83.0882)
Valid: 21 [ 100/390]  Loss: 2.102 (1.77)  Acc@1: 53.1250 (54.5637)  Acc@5: 78.1250 (82.5495)
Valid: 21 [ 150/390]  Loss: 1.438 (1.75)  Acc@1: 54.6875 (54.6668)  Acc@5: 84.3750 (82.6883)
Valid: 21 [ 200/390]  Loss: 1.843 (1.77)  Acc@1: 45.3125 (54.1744)  Acc@5: 81.2500 (82.7581)
Valid: 21 [ 250/390]  Loss: 1.491 (1.75)  Acc@1: 64.0625 (54.2891)  Acc@5: 87.5000 (82.9432)
Valid: 21 [ 300/390]  Loss: 2.469 (1.76)  Acc@1: 43.7500 (54.1684)  Acc@5: 75.0000 (82.7606)
Valid: 21 [ 350/390]  Loss: 2.007 (1.77)  Acc@1: 48.4375 (53.9842)  Acc@5: 79.6875 (82.5632)
Valid: 21 [ 390/390]  Loss: 1.784 (1.76)  Acc@1: 57.5000 (54.1080)  Acc@5: 82.5000 (82.7160)
valid_acc 54.108000
epoch = 21   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2497, 0.7503],
        [0.2767, 0.7233],
        [0.2634, 0.7366],
        [0.3686, 0.6314],
        [0.2544, 0.7456],
        [0.3184, 0.6816],
        [0.4031, 0.5969],
        [0.2444, 0.7556],
        [0.1835, 0.8165],
        [0.2885, 0.7115],
        [0.3408, 0.6592],
        [0.2844, 0.7156],
        [0.1677, 0.8323],
        [0.1300, 0.8700]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3435, 0.6565],
        [0.3354, 0.6646],
        [0.2971, 0.7029],
        [0.3649, 0.6351],
        [0.2907, 0.7093],
        [0.3849, 0.6151],
        [0.2922, 0.7078],
        [0.2968, 0.7032],
        [0.3388, 0.6612],
        [0.4326, 0.5674],
        [0.3563, 0.6437],
        [0.3219, 0.6781],
        [0.4071, 0.5929],
        [0.3720, 0.6280]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 22 [   0/390]  Loss: 0.7657 (0.766)  Acc@1: 73.4375 (73.4375)  Acc@5: 96.8750 (96.8750)LR: 1.525e-02
Train: 22 [  50/390]  Loss: 1.235 (1.06)  Acc@1: 64.0625 (68.4130)  Acc@5: 89.0625 (92.5245)LR: 1.525e-02
Train: 22 [ 100/390]  Loss: 1.010 (1.06)  Acc@1: 67.1875 (68.5334)  Acc@5: 92.1875 (92.5588)LR: 1.525e-02
Train: 22 [ 150/390]  Loss: 1.331 (1.05)  Acc@1: 60.9375 (68.6155)  Acc@5: 89.0625 (92.5911)LR: 1.525e-02
Train: 22 [ 200/390]  Loss: 0.9605 (1.06)  Acc@1: 68.7500 (68.5168)  Acc@5: 93.7500 (92.5373)LR: 1.525e-02
Train: 22 [ 250/390]  Loss: 1.289 (1.07)  Acc@1: 60.9375 (68.2333)  Acc@5: 87.5000 (92.3307)LR: 1.525e-02
Train: 22 [ 300/390]  Loss: 1.431 (1.09)  Acc@1: 60.9375 (67.7014)  Acc@5: 85.9375 (92.0110)LR: 1.525e-02
Train: 22 [ 350/390]  Loss: 0.6765 (1.10)  Acc@1: 76.5625 (67.5214)  Acc@5: 96.8750 (91.8892)LR: 1.525e-02
Train: 22 [ 390/390]  Loss: 1.139 (1.10)  Acc@1: 60.0000 (67.4040)  Acc@5: 92.5000 (91.9120)LR: 1.525e-02
train_acc 67.404000
Valid: 22 [   0/390]  Loss: 1.433 (1.43)  Acc@1: 56.2500 (56.2500)  Acc@5: 90.6250 (90.6250)
Valid: 22 [  50/390]  Loss: 1.804 (1.74)  Acc@1: 54.6875 (54.0748)  Acc@5: 79.6875 (83.0882)
Valid: 22 [ 100/390]  Loss: 1.693 (1.71)  Acc@1: 50.0000 (55.4146)  Acc@5: 85.9375 (83.9109)
Valid: 22 [ 150/390]  Loss: 1.655 (1.70)  Acc@1: 53.1250 (55.5671)  Acc@5: 81.2500 (83.8473)
Valid: 22 [ 200/390]  Loss: 1.484 (1.70)  Acc@1: 59.3750 (55.8458)  Acc@5: 84.3750 (84.0407)
Valid: 22 [ 250/390]  Loss: 1.810 (1.70)  Acc@1: 60.9375 (55.7395)  Acc@5: 84.3750 (84.1011)
Valid: 22 [ 300/390]  Loss: 1.932 (1.70)  Acc@1: 50.0000 (55.5336)  Acc@5: 81.2500 (83.9961)
Valid: 22 [ 350/390]  Loss: 1.562 (1.71)  Acc@1: 57.8125 (55.3775)  Acc@5: 82.8125 (83.9254)
Valid: 22 [ 390/390]  Loss: 1.507 (1.71)  Acc@1: 52.5000 (55.4040)  Acc@5: 92.5000 (83.8520)
valid_acc 55.404000
epoch = 22   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2428, 0.7572],
        [0.2700, 0.7300],
        [0.2589, 0.7411],
        [0.3708, 0.6292],
        [0.2530, 0.7470],
        [0.3189, 0.6811],
        [0.4106, 0.5894],
        [0.2395, 0.7605],
        [0.1756, 0.8244],
        [0.2857, 0.7143],
        [0.3474, 0.6526],
        [0.2844, 0.7156],
        [0.1567, 0.8433],
        [0.1207, 0.8793]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3401, 0.6599],
        [0.3287, 0.6713],
        [0.2903, 0.7097],
        [0.3546, 0.6454],
        [0.2849, 0.7151],
        [0.3829, 0.6171],
        [0.2805, 0.7195],
        [0.2933, 0.7067],
        [0.3359, 0.6641],
        [0.4329, 0.5671],
        [0.3509, 0.6491],
        [0.3183, 0.6817],
        [0.4018, 0.5982],
        [0.3653, 0.6347]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 23 [   0/390]  Loss: 1.085 (1.09)  Acc@1: 73.4375 (73.4375)  Acc@5: 87.5000 (87.5000)LR: 1.450e-02
Train: 23 [  50/390]  Loss: 0.8118 (0.972)  Acc@1: 76.5625 (71.3235)  Acc@5: 98.4375 (93.9645)LR: 1.450e-02
Train: 23 [ 100/390]  Loss: 1.013 (0.977)  Acc@1: 64.0625 (71.2098)  Acc@5: 92.1875 (93.8583)LR: 1.450e-02
Train: 23 [ 150/390]  Loss: 1.314 (1.00)  Acc@1: 67.1875 (70.2194)  Acc@5: 90.6250 (93.6776)LR: 1.450e-02
Train: 23 [ 200/390]  Loss: 0.8219 (1.01)  Acc@1: 70.3125 (69.8383)  Acc@5: 93.7500 (93.4313)LR: 1.450e-02
Train: 23 [ 250/390]  Loss: 1.216 (1.02)  Acc@1: 70.3125 (69.5095)  Acc@5: 89.0625 (93.2582)LR: 1.450e-02
Train: 23 [ 300/390]  Loss: 0.8180 (1.03)  Acc@1: 75.0000 (69.1289)  Acc@5: 95.3125 (93.1426)LR: 1.450e-02
Train: 23 [ 350/390]  Loss: 1.213 (1.05)  Acc@1: 67.1875 (68.9859)  Acc@5: 89.0625 (92.9087)LR: 1.450e-02
Train: 23 [ 390/390]  Loss: 0.9031 (1.06)  Acc@1: 65.0000 (68.7000)  Acc@5: 97.5000 (92.7760)LR: 1.450e-02
train_acc 68.700000
Valid: 23 [   0/390]  Loss: 1.131 (1.13)  Acc@1: 64.0625 (64.0625)  Acc@5: 93.7500 (93.7500)
Valid: 23 [  50/390]  Loss: 1.914 (1.68)  Acc@1: 59.3750 (56.3113)  Acc@5: 81.2500 (84.2831)
Valid: 23 [ 100/390]  Loss: 1.587 (1.66)  Acc@1: 62.5000 (56.5285)  Acc@5: 85.9375 (84.5606)
Valid: 23 [ 150/390]  Loss: 1.434 (1.67)  Acc@1: 62.5000 (56.2707)  Acc@5: 82.8125 (84.4474)
Valid: 23 [ 200/390]  Loss: 1.701 (1.68)  Acc@1: 53.1250 (56.3044)  Acc@5: 85.9375 (84.5382)
Valid: 23 [ 250/390]  Loss: 1.576 (1.68)  Acc@1: 57.8125 (56.1815)  Acc@5: 81.2500 (84.4684)
Valid: 23 [ 300/390]  Loss: 1.899 (1.68)  Acc@1: 50.0000 (56.1566)  Acc@5: 81.2500 (84.4684)
Valid: 23 [ 350/390]  Loss: 1.984 (1.69)  Acc@1: 51.5625 (55.8538)  Acc@5: 81.2500 (84.3884)
Valid: 23 [ 390/390]  Loss: 1.941 (1.68)  Acc@1: 55.0000 (55.8960)  Acc@5: 80.0000 (84.3520)
valid_acc 55.896000
epoch = 23   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2339, 0.7661],
        [0.2638, 0.7362],
        [0.2525, 0.7475],
        [0.3711, 0.6289],
        [0.2493, 0.7507],
        [0.3182, 0.6818],
        [0.4128, 0.5872],
        [0.2360, 0.7640],
        [0.1678, 0.8322],
        [0.2830, 0.7170],
        [0.3554, 0.6446],
        [0.2863, 0.7137],
        [0.1488, 0.8512],
        [0.1116, 0.8884]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3396, 0.6604],
        [0.3194, 0.6806],
        [0.2878, 0.7122],
        [0.3508, 0.6492],
        [0.2785, 0.7215],
        [0.3848, 0.6152],
        [0.2736, 0.7264],
        [0.2835, 0.7165],
        [0.3314, 0.6686],
        [0.4318, 0.5682],
        [0.3490, 0.6510],
        [0.3141, 0.6859],
        [0.3986, 0.6014],
        [0.3551, 0.6449]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 24 [   0/390]  Loss: 0.8747 (0.875)  Acc@1: 75.0000 (75.0000)  Acc@5: 93.7500 (93.7500)LR: 1.375e-02
Train: 24 [  50/390]  Loss: 1.093 (0.974)  Acc@1: 68.7500 (71.2316)  Acc@5: 93.7500 (93.6275)LR: 1.375e-02
Train: 24 [ 100/390]  Loss: 0.9924 (0.965)  Acc@1: 68.7500 (71.1170)  Acc@5: 90.6250 (94.0439)LR: 1.375e-02
Train: 24 [ 150/390]  Loss: 0.9765 (0.992)  Acc@1: 70.3125 (70.3642)  Acc@5: 92.1875 (93.5948)LR: 1.375e-02
Train: 24 [ 200/390]  Loss: 1.233 (0.994)  Acc@1: 65.6250 (70.3902)  Acc@5: 90.6250 (93.5557)LR: 1.375e-02
Train: 24 [ 250/390]  Loss: 1.074 (1.00)  Acc@1: 62.5000 (70.0012)  Acc@5: 95.3125 (93.4387)LR: 1.375e-02
Train: 24 [ 300/390]  Loss: 0.8234 (1.00)  Acc@1: 76.5625 (69.9699)  Acc@5: 95.3125 (93.4022)LR: 1.375e-02
Train: 24 [ 350/390]  Loss: 1.051 (1.00)  Acc@1: 60.9375 (70.0855)  Acc@5: 93.7500 (93.3716)LR: 1.375e-02
Train: 24 [ 390/390]  Loss: 0.7564 (1.01)  Acc@1: 85.0000 (69.9960)  Acc@5: 92.5000 (93.2760)LR: 1.375e-02
train_acc 69.996000
Valid: 24 [   0/390]  Loss: 1.464 (1.46)  Acc@1: 67.1875 (67.1875)  Acc@5: 89.0625 (89.0625)
Valid: 24 [  50/390]  Loss: 1.523 (1.78)  Acc@1: 65.6250 (55.0551)  Acc@5: 84.3750 (83.5478)
Valid: 24 [ 100/390]  Loss: 1.760 (1.78)  Acc@1: 57.8125 (54.8731)  Acc@5: 81.2500 (83.2921)
Valid: 24 [ 150/390]  Loss: 1.991 (1.77)  Acc@1: 50.0000 (55.0186)  Acc@5: 82.8125 (83.4127)
Valid: 24 [ 200/390]  Loss: 1.436 (1.76)  Acc@1: 60.9375 (55.0529)  Acc@5: 90.6250 (83.5277)
Valid: 24 [ 250/390]  Loss: 1.861 (1.76)  Acc@1: 56.2500 (54.9490)  Acc@5: 76.5625 (83.5782)
Valid: 24 [ 300/390]  Loss: 1.570 (1.76)  Acc@1: 64.0625 (54.9730)  Acc@5: 84.3750 (83.5808)
Valid: 24 [ 350/390]  Loss: 1.695 (1.76)  Acc@1: 50.0000 (54.8789)  Acc@5: 85.9375 (83.5292)
Valid: 24 [ 390/390]  Loss: 1.620 (1.76)  Acc@1: 52.5000 (54.8800)  Acc@5: 85.0000 (83.5720)
valid_acc 54.880000
epoch = 24   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2242, 0.7758],
        [0.2553, 0.7447],
        [0.2448, 0.7552],
        [0.3694, 0.6306],
        [0.2469, 0.7531],
        [0.3152, 0.6848],
        [0.4167, 0.5833],
        [0.2341, 0.7659],
        [0.1605, 0.8395],
        [0.2771, 0.7229],
        [0.3599, 0.6401],
        [0.2841, 0.7159],
        [0.1411, 0.8589],
        [0.1032, 0.8968]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3362, 0.6638],
        [0.3130, 0.6870],
        [0.2834, 0.7166],
        [0.3449, 0.6551],
        [0.2740, 0.7260],
        [0.3855, 0.6145],
        [0.2614, 0.7386],
        [0.2788, 0.7212],
        [0.3279, 0.6721],
        [0.4287, 0.5713],
        [0.3407, 0.6593],
        [0.3075, 0.6925],
        [0.3911, 0.6089],
        [0.3476, 0.6524]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 25 [   0/390]  Loss: 1.091 (1.09)  Acc@1: 71.8750 (71.8750)  Acc@5: 89.0625 (89.0625)LR: 1.300e-02
Train: 25 [  50/390]  Loss: 0.6970 (0.949)  Acc@1: 78.1250 (71.4154)  Acc@5: 98.4375 (93.9951)LR: 1.300e-02
Train: 25 [ 100/390]  Loss: 0.4897 (0.905)  Acc@1: 89.0625 (72.6949)  Acc@5: 98.4375 (94.6937)LR: 1.300e-02
Train: 25 [ 150/390]  Loss: 0.9112 (0.913)  Acc@1: 70.3125 (72.1130)  Acc@5: 93.7500 (94.6502)LR: 1.300e-02
Train: 25 [ 200/390]  Loss: 1.306 (0.936)  Acc@1: 64.0625 (71.7273)  Acc@5: 87.5000 (94.2864)LR: 1.300e-02
Train: 25 [ 250/390]  Loss: 0.7234 (0.943)  Acc@1: 73.4375 (71.6447)  Acc@5: 100.0000 (94.1982)LR: 1.300e-02
Train: 25 [ 300/390]  Loss: 1.060 (0.962)  Acc@1: 65.6250 (71.1275)  Acc@5: 93.7500 (93.9161)LR: 1.300e-02
Train: 25 [ 350/390]  Loss: 0.9479 (0.965)  Acc@1: 71.8750 (70.9535)  Acc@5: 96.8750 (93.9236)LR: 1.300e-02
Train: 25 [ 390/390]  Loss: 1.214 (0.969)  Acc@1: 65.0000 (70.8880)  Acc@5: 90.0000 (93.8120)LR: 1.300e-02
train_acc 70.888000
Valid: 25 [   0/390]  Loss: 2.179 (2.18)  Acc@1: 48.4375 (48.4375)  Acc@5: 78.1250 (78.1250)
Valid: 25 [  50/390]  Loss: 1.510 (1.70)  Acc@1: 54.6875 (56.1887)  Acc@5: 87.5000 (84.0380)
Valid: 25 [ 100/390]  Loss: 1.387 (1.70)  Acc@1: 62.5000 (55.9715)  Acc@5: 84.3750 (84.3441)
Valid: 25 [ 150/390]  Loss: 1.668 (1.70)  Acc@1: 51.5625 (56.1983)  Acc@5: 89.0625 (84.4267)
Valid: 25 [ 200/390]  Loss: 1.654 (1.71)  Acc@1: 54.6875 (56.3200)  Acc@5: 87.5000 (84.4139)
Valid: 25 [ 250/390]  Loss: 1.284 (1.70)  Acc@1: 75.0000 (56.2811)  Acc@5: 87.5000 (84.4684)
Valid: 25 [ 300/390]  Loss: 1.903 (1.71)  Acc@1: 53.1250 (56.0735)  Acc@5: 85.9375 (84.4165)
Valid: 25 [ 350/390]  Loss: 1.975 (1.70)  Acc@1: 59.3750 (56.3079)  Acc@5: 79.6875 (84.4863)
Valid: 25 [ 390/390]  Loss: 1.252 (1.70)  Acc@1: 65.0000 (56.3760)  Acc@5: 90.0000 (84.5240)
valid_acc 56.376000
epoch = 25   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2164, 0.7836],
        [0.2481, 0.7519],
        [0.2434, 0.7566],
        [0.3686, 0.6314],
        [0.2476, 0.7524],
        [0.3164, 0.6836],
        [0.4213, 0.5787],
        [0.2311, 0.7689],
        [0.1532, 0.8468],
        [0.2738, 0.7262],
        [0.3657, 0.6343],
        [0.2860, 0.7140],
        [0.1343, 0.8657],
        [0.0954, 0.9046]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3335, 0.6665],
        [0.3042, 0.6958],
        [0.2755, 0.7245],
        [0.3380, 0.6620],
        [0.2701, 0.7299],
        [0.3830, 0.6170],
        [0.2539, 0.7461],
        [0.2764, 0.7236],
        [0.3271, 0.6729],
        [0.4250, 0.5750],
        [0.3426, 0.6574],
        [0.3041, 0.6959],
        [0.3874, 0.6126],
        [0.3384, 0.6616]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 26 [   0/390]  Loss: 1.022 (1.02)  Acc@1: 75.0000 (75.0000)  Acc@5: 92.1875 (92.1875)LR: 1.225e-02
Train: 26 [  50/390]  Loss: 0.7793 (0.838)  Acc@1: 81.2500 (75.9191)  Acc@5: 93.7500 (94.8223)LR: 1.225e-02
Train: 26 [ 100/390]  Loss: 0.7201 (0.848)  Acc@1: 78.1250 (74.8917)  Acc@5: 98.4375 (95.0031)LR: 1.225e-02
Train: 26 [ 150/390]  Loss: 1.097 (0.867)  Acc@1: 70.3125 (74.0791)  Acc@5: 93.7500 (94.9710)LR: 1.225e-02
Train: 26 [ 200/390]  Loss: 1.041 (0.877)  Acc@1: 68.7500 (73.6552)  Acc@5: 95.3125 (94.8461)LR: 1.225e-02
Train: 26 [ 250/390]  Loss: 0.9973 (0.888)  Acc@1: 67.1875 (73.3566)  Acc@5: 95.3125 (94.7585)LR: 1.225e-02
Train: 26 [ 300/390]  Loss: 0.9236 (0.894)  Acc@1: 68.7500 (73.0430)  Acc@5: 92.1875 (94.7259)LR: 1.225e-02
Train: 26 [ 350/390]  Loss: 1.033 (0.903)  Acc@1: 64.0625 (72.7386)  Acc@5: 95.3125 (94.5735)LR: 1.225e-02
Train: 26 [ 390/390]  Loss: 1.143 (0.913)  Acc@1: 62.5000 (72.4120)  Acc@5: 92.5000 (94.4840)LR: 1.225e-02
train_acc 72.412000
Valid: 26 [   0/390]  Loss: 2.195 (2.19)  Acc@1: 50.0000 (50.0000)  Acc@5: 73.4375 (73.4375)
Valid: 26 [  50/390]  Loss: 1.564 (1.67)  Acc@1: 57.8125 (56.6483)  Acc@5: 89.0625 (85.1716)
Valid: 26 [ 100/390]  Loss: 2.329 (1.67)  Acc@1: 51.5625 (57.2556)  Acc@5: 73.4375 (85.0402)
Valid: 26 [ 150/390]  Loss: 1.844 (1.69)  Acc@1: 46.8750 (56.8709)  Acc@5: 82.8125 (84.7579)
Valid: 26 [ 200/390]  Loss: 1.980 (1.66)  Acc@1: 56.2500 (57.1440)  Acc@5: 84.3750 (85.1057)
Valid: 26 [ 250/390]  Loss: 1.550 (1.68)  Acc@1: 60.9375 (56.8725)  Acc@5: 87.5000 (84.9228)
Valid: 26 [ 300/390]  Loss: 1.625 (1.67)  Acc@1: 54.6875 (56.7639)  Acc@5: 90.6250 (84.8889)
Valid: 26 [ 350/390]  Loss: 2.009 (1.69)  Acc@1: 48.4375 (56.4637)  Acc@5: 79.6875 (84.7712)
Valid: 26 [ 390/390]  Loss: 1.457 (1.69)  Acc@1: 60.0000 (56.5200)  Acc@5: 85.0000 (84.7080)
valid_acc 56.520000
epoch = 26   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2112, 0.7888],
        [0.2433, 0.7567],
        [0.2436, 0.7564],
        [0.3724, 0.6276],
        [0.2450, 0.7550],
        [0.3143, 0.6857],
        [0.4284, 0.5716],
        [0.2300, 0.7700],
        [0.1473, 0.8527],
        [0.2719, 0.7281],
        [0.3741, 0.6259],
        [0.2858, 0.7142],
        [0.1287, 0.8713],
        [0.0877, 0.9123]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3331, 0.6669],
        [0.2967, 0.7033],
        [0.2679, 0.7321],
        [0.3320, 0.6680],
        [0.2667, 0.7333],
        [0.3872, 0.6128],
        [0.2472, 0.7528],
        [0.2737, 0.7263],
        [0.3281, 0.6719],
        [0.4234, 0.5766],
        [0.3419, 0.6581],
        [0.2979, 0.7021],
        [0.3806, 0.6194],
        [0.3341, 0.6659]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 27 [   0/390]  Loss: 0.6879 (0.688)  Acc@1: 82.8125 (82.8125)  Acc@5: 93.7500 (93.7500)LR: 1.150e-02
Train: 27 [  50/390]  Loss: 0.7714 (0.826)  Acc@1: 73.4375 (75.6127)  Acc@5: 96.8750 (95.3431)LR: 1.150e-02
Train: 27 [ 100/390]  Loss: 0.7833 (0.832)  Acc@1: 75.0000 (75.1856)  Acc@5: 93.7500 (95.2970)LR: 1.150e-02
Train: 27 [ 150/390]  Loss: 0.7704 (0.833)  Acc@1: 71.8750 (74.8655)  Acc@5: 96.8750 (95.3746)LR: 1.150e-02
Train: 27 [ 200/390]  Loss: 0.8774 (0.833)  Acc@1: 70.3125 (74.8523)  Acc@5: 100.0000 (95.4757)LR: 1.150e-02
Train: 27 [ 250/390]  Loss: 1.055 (0.848)  Acc@1: 71.8750 (74.3650)  Acc@5: 93.7500 (95.3499)LR: 1.150e-02
Train: 27 [ 300/390]  Loss: 0.7337 (0.859)  Acc@1: 81.2500 (74.1279)  Acc@5: 96.8750 (95.1827)LR: 1.150e-02
Train: 27 [ 350/390]  Loss: 0.4205 (0.861)  Acc@1: 92.1875 (74.1141)  Acc@5: 98.4375 (95.0365)LR: 1.150e-02
Train: 27 [ 390/390]  Loss: 1.064 (0.871)  Acc@1: 65.0000 (73.8320)  Acc@5: 90.0000 (94.8880)LR: 1.150e-02
train_acc 73.832000
Valid: 27 [   0/390]  Loss: 1.240 (1.24)  Acc@1: 73.4375 (73.4375)  Acc@5: 89.0625 (89.0625)
Valid: 27 [  50/390]  Loss: 1.603 (1.66)  Acc@1: 54.6875 (57.3223)  Acc@5: 85.9375 (85.1409)
Valid: 27 [ 100/390]  Loss: 1.283 (1.63)  Acc@1: 64.0625 (57.7506)  Acc@5: 89.0625 (85.2413)
Valid: 27 [ 150/390]  Loss: 1.479 (1.63)  Acc@1: 59.3750 (57.5642)  Acc@5: 82.8125 (85.0373)
Valid: 27 [ 200/390]  Loss: 1.906 (1.63)  Acc@1: 56.2500 (57.5871)  Acc@5: 79.6875 (85.2146)
Valid: 27 [ 250/390]  Loss: 1.586 (1.63)  Acc@1: 62.5000 (57.4265)  Acc@5: 87.5000 (85.1345)
Valid: 27 [ 300/390]  Loss: 1.611 (1.63)  Acc@1: 54.6875 (57.3816)  Acc@5: 92.1875 (85.1952)
Valid: 27 [ 350/390]  Loss: 1.453 (1.64)  Acc@1: 68.7500 (57.4786)  Acc@5: 87.5000 (85.1318)
Valid: 27 [ 390/390]  Loss: 2.183 (1.64)  Acc@1: 37.5000 (57.3120)  Acc@5: 82.5000 (85.1600)
valid_acc 57.312000
epoch = 27   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2053, 0.7947],
        [0.2393, 0.7607],
        [0.2397, 0.7603],
        [0.3746, 0.6254],
        [0.2451, 0.7549],
        [0.3175, 0.6825],
        [0.4365, 0.5635],
        [0.2318, 0.7682],
        [0.1443, 0.8557],
        [0.2712, 0.7288],
        [0.3779, 0.6221],
        [0.2874, 0.7126],
        [0.1227, 0.8773],
        [0.0809, 0.9191]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3322, 0.6678],
        [0.2954, 0.7046],
        [0.2617, 0.7383],
        [0.3284, 0.6716],
        [0.2644, 0.7356],
        [0.3897, 0.6103],
        [0.2450, 0.7550],
        [0.2710, 0.7290],
        [0.3257, 0.6743],
        [0.4252, 0.5748],
        [0.3423, 0.6577],
        [0.2933, 0.7067],
        [0.3754, 0.6246],
        [0.3271, 0.6729]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 28 [   0/390]  Loss: 0.7560 (0.756)  Acc@1: 73.4375 (73.4375)  Acc@5: 95.3125 (95.3125)LR: 1.075e-02
Train: 28 [  50/390]  Loss: 0.6438 (0.742)  Acc@1: 79.6875 (77.6654)  Acc@5: 95.3125 (96.2316)LR: 1.075e-02
Train: 28 [ 100/390]  Loss: 0.6453 (0.786)  Acc@1: 78.1250 (76.1139)  Acc@5: 96.8750 (95.8849)LR: 1.075e-02
Train: 28 [ 150/390]  Loss: 0.7671 (0.803)  Acc@1: 71.8750 (75.2897)  Acc@5: 96.8750 (95.7781)LR: 1.075e-02
Train: 28 [ 200/390]  Loss: 1.254 (0.806)  Acc@1: 64.0625 (75.1632)  Acc@5: 89.0625 (95.7167)LR: 1.075e-02
Train: 28 [ 250/390]  Loss: 0.7373 (0.807)  Acc@1: 81.2500 (75.2801)  Acc@5: 96.8750 (95.7234)LR: 1.075e-02
Train: 28 [ 300/390]  Loss: 1.027 (0.816)  Acc@1: 65.6250 (75.0000)  Acc@5: 93.7500 (95.6136)LR: 1.075e-02
Train: 28 [ 350/390]  Loss: 0.8232 (0.828)  Acc@1: 76.5625 (74.7952)  Acc@5: 96.8750 (95.4906)LR: 1.075e-02
Train: 28 [ 390/390]  Loss: 0.9931 (0.834)  Acc@1: 67.5000 (74.6920)  Acc@5: 90.0000 (95.3680)LR: 1.075e-02
train_acc 74.692000
Valid: 28 [   0/390]  Loss: 1.723 (1.72)  Acc@1: 53.1250 (53.1250)  Acc@5: 81.2500 (81.2500)
Valid: 28 [  50/390]  Loss: 1.653 (1.64)  Acc@1: 54.6875 (56.8321)  Acc@5: 84.3750 (85.1716)
Valid: 28 [ 100/390]  Loss: 2.146 (1.61)  Acc@1: 46.8750 (57.8744)  Acc@5: 73.4375 (85.6126)
Valid: 28 [ 150/390]  Loss: 2.116 (1.64)  Acc@1: 43.7500 (57.5642)  Acc@5: 84.3750 (85.2959)
Valid: 28 [ 200/390]  Loss: 1.410 (1.64)  Acc@1: 60.9375 (57.4471)  Acc@5: 87.5000 (85.5333)
Valid: 28 [ 250/390]  Loss: 1.553 (1.64)  Acc@1: 56.2500 (57.3394)  Acc@5: 87.5000 (85.5764)
Valid: 28 [ 300/390]  Loss: 1.353 (1.63)  Acc@1: 60.9375 (57.5478)  Acc@5: 89.0625 (85.6468)
Valid: 28 [ 350/390]  Loss: 1.784 (1.63)  Acc@1: 53.1250 (57.4964)  Acc@5: 82.8125 (85.5858)
Valid: 28 [ 390/390]  Loss: 1.662 (1.62)  Acc@1: 57.5000 (57.6600)  Acc@5: 82.5000 (85.6240)
valid_acc 57.660000
epoch = 28   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2001, 0.7999],
        [0.2374, 0.7626],
        [0.2387, 0.7613],
        [0.3735, 0.6265],
        [0.2403, 0.7597],
        [0.3192, 0.6808],
        [0.4419, 0.5581],
        [0.2338, 0.7662],
        [0.1422, 0.8578],
        [0.2694, 0.7306],
        [0.3863, 0.6137],
        [0.2904, 0.7096],
        [0.1172, 0.8828],
        [0.0752, 0.9248]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3259, 0.6741],
        [0.2865, 0.7135],
        [0.2556, 0.7444],
        [0.3230, 0.6770],
        [0.2632, 0.7368],
        [0.3858, 0.6142],
        [0.2385, 0.7615],
        [0.2670, 0.7330],
        [0.3206, 0.6794],
        [0.4262, 0.5738],
        [0.3413, 0.6587],
        [0.2936, 0.7064],
        [0.3680, 0.6320],
        [0.3166, 0.6834]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 29 [   0/390]  Loss: 0.6681 (0.668)  Acc@1: 78.1250 (78.1250)  Acc@5: 95.3125 (95.3125)LR: 1.002e-02
Train: 29 [  50/390]  Loss: 0.8295 (0.710)  Acc@1: 71.8750 (78.4007)  Acc@5: 93.7500 (96.8137)LR: 1.002e-02
Train: 29 [ 100/390]  Loss: 0.8869 (0.725)  Acc@1: 73.4375 (78.1250)  Acc@5: 95.3125 (96.5656)LR: 1.002e-02
Train: 29 [ 150/390]  Loss: 0.5865 (0.730)  Acc@1: 81.2500 (78.0526)  Acc@5: 98.4375 (96.5335)LR: 1.002e-02
Train: 29 [ 200/390]  Loss: 0.9277 (0.742)  Acc@1: 70.3125 (77.6042)  Acc@5: 95.3125 (96.3775)LR: 1.002e-02
Train: 29 [ 250/390]  Loss: 0.8127 (0.756)  Acc@1: 73.4375 (77.2908)  Acc@5: 95.3125 (96.2463)LR: 1.002e-02
Train: 29 [ 300/390]  Loss: 0.7532 (0.764)  Acc@1: 75.0000 (77.0816)  Acc@5: 98.4375 (96.2157)LR: 1.002e-02
Train: 29 [ 350/390]  Loss: 0.7049 (0.771)  Acc@1: 78.1250 (76.7272)  Acc@5: 96.8750 (96.0826)LR: 1.002e-02
Train: 29 [ 390/390]  Loss: 1.312 (0.782)  Acc@1: 62.5000 (76.4200)  Acc@5: 87.5000 (95.9880)LR: 1.002e-02
train_acc 76.420000
Valid: 29 [   0/390]  Loss: 1.842 (1.84)  Acc@1: 53.1250 (53.1250)  Acc@5: 81.2500 (81.2500)
Valid: 29 [  50/390]  Loss: 1.596 (1.64)  Acc@1: 59.3750 (57.8431)  Acc@5: 85.9375 (85.5392)
Valid: 29 [ 100/390]  Loss: 1.768 (1.64)  Acc@1: 57.8125 (58.6015)  Acc@5: 84.3750 (85.2413)
Valid: 29 [ 150/390]  Loss: 1.999 (1.64)  Acc@1: 53.1250 (58.9197)  Acc@5: 82.8125 (85.2028)
Valid: 29 [ 200/390]  Loss: 1.456 (1.65)  Acc@1: 60.9375 (58.4111)  Acc@5: 85.9375 (85.2223)
Valid: 29 [ 250/390]  Loss: 1.476 (1.66)  Acc@1: 46.8750 (58.2234)  Acc@5: 85.9375 (85.1531)
Valid: 29 [ 300/390]  Loss: 1.880 (1.64)  Acc@1: 48.4375 (58.4250)  Acc@5: 78.1250 (85.2575)
Valid: 29 [ 350/390]  Loss: 1.674 (1.65)  Acc@1: 60.9375 (58.2577)  Acc@5: 85.9375 (85.2119)
Valid: 29 [ 390/390]  Loss: 1.476 (1.64)  Acc@1: 60.0000 (58.1840)  Acc@5: 87.5000 (85.3080)
valid_acc 58.184000
epoch = 29   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1922, 0.8078],
        [0.2316, 0.7684],
        [0.2349, 0.7651],
        [0.3777, 0.6223],
        [0.2365, 0.7635],
        [0.3215, 0.6785],
        [0.4468, 0.5532],
        [0.2372, 0.7628],
        [0.1371, 0.8629],
        [0.2679, 0.7321],
        [0.3952, 0.6048],
        [0.2939, 0.7061],
        [0.1111, 0.8889],
        [0.0701, 0.9299]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3256, 0.6744],
        [0.2794, 0.7206],
        [0.2484, 0.7516],
        [0.3179, 0.6821],
        [0.2589, 0.7411],
        [0.3826, 0.6174],
        [0.2366, 0.7634],
        [0.2679, 0.7321],
        [0.3217, 0.6783],
        [0.4295, 0.5705],
        [0.3417, 0.6583],
        [0.2879, 0.7121],
        [0.3657, 0.6343],
        [0.3095, 0.6905]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 30 [   0/390]  Loss: 0.7200 (0.720)  Acc@1: 76.5625 (76.5625)  Acc@5: 95.3125 (95.3125)LR: 9.292e-03
Train: 30 [  50/390]  Loss: 0.5418 (0.697)  Acc@1: 79.6875 (78.7990)  Acc@5: 100.0000 (97.0282)LR: 9.292e-03
Train: 30 [ 100/390]  Loss: 0.6711 (0.712)  Acc@1: 73.4375 (78.3880)  Acc@5: 96.8750 (96.8131)LR: 9.292e-03
Train: 30 [ 150/390]  Loss: 0.7536 (0.709)  Acc@1: 81.2500 (78.3837)  Acc@5: 95.3125 (96.8543)LR: 9.292e-03
Train: 30 [ 200/390]  Loss: 0.6712 (0.720)  Acc@1: 78.1250 (78.0706)  Acc@5: 96.8750 (96.6340)LR: 9.292e-03
Train: 30 [ 250/390]  Loss: 0.6681 (0.728)  Acc@1: 81.2500 (77.7951)  Acc@5: 95.3125 (96.6260)LR: 9.292e-03
Train: 30 [ 300/390]  Loss: 0.9189 (0.738)  Acc@1: 73.4375 (77.5384)  Acc@5: 93.7500 (96.4182)LR: 9.292e-03
Train: 30 [ 350/390]  Loss: 0.7701 (0.746)  Acc@1: 76.5625 (77.1946)  Acc@5: 96.8750 (96.4343)LR: 9.292e-03
Train: 30 [ 390/390]  Loss: 0.7558 (0.747)  Acc@1: 75.0000 (77.0920)  Acc@5: 97.5000 (96.3960)LR: 9.292e-03
train_acc 77.092000
Valid: 30 [   0/390]  Loss: 1.984 (1.98)  Acc@1: 59.3750 (59.3750)  Acc@5: 79.6875 (79.6875)
Valid: 30 [  50/390]  Loss: 1.966 (1.62)  Acc@1: 53.1250 (59.6507)  Acc@5: 81.2500 (86.1826)
Valid: 30 [ 100/390]  Loss: 1.804 (1.64)  Acc@1: 50.0000 (58.9264)  Acc@5: 82.8125 (85.7673)
Valid: 30 [ 150/390]  Loss: 1.561 (1.63)  Acc@1: 62.5000 (58.9921)  Acc@5: 84.3750 (85.8961)
Valid: 30 [ 200/390]  Loss: 1.378 (1.61)  Acc@1: 68.7500 (59.4061)  Acc@5: 89.0625 (85.9997)
Valid: 30 [ 250/390]  Loss: 1.392 (1.60)  Acc@1: 64.0625 (59.6116)  Acc@5: 89.0625 (86.1305)
Valid: 30 [ 300/390]  Loss: 1.895 (1.60)  Acc@1: 59.3750 (59.5411)  Acc@5: 76.5625 (86.0673)
Valid: 30 [ 350/390]  Loss: 1.322 (1.61)  Acc@1: 64.0625 (59.3661)  Acc@5: 85.9375 (85.9241)
Valid: 30 [ 390/390]  Loss: 1.809 (1.61)  Acc@1: 60.0000 (59.2080)  Acc@5: 82.5000 (85.9480)
valid_acc 59.208000
epoch = 30   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1838, 0.8162],
        [0.2279, 0.7721],
        [0.2321, 0.7679],
        [0.3804, 0.6196],
        [0.2347, 0.7653],
        [0.3234, 0.6766],
        [0.4518, 0.5482],
        [0.2392, 0.7608],
        [0.1354, 0.8646],
        [0.2683, 0.7317],
        [0.4070, 0.5930],
        [0.2994, 0.7006],
        [0.1092, 0.8908],
        [0.0660, 0.9340]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3217, 0.6783],
        [0.2723, 0.7277],
        [0.2420, 0.7580],
        [0.3142, 0.6858],
        [0.2567, 0.7433],
        [0.3835, 0.6165],
        [0.2318, 0.7682],
        [0.2657, 0.7343],
        [0.3162, 0.6838],
        [0.4264, 0.5736],
        [0.3398, 0.6602],
        [0.2798, 0.7202],
        [0.3631, 0.6369],
        [0.3049, 0.6951]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 31 [   0/390]  Loss: 0.6332 (0.633)  Acc@1: 82.8125 (82.8125)  Acc@5: 95.3125 (95.3125)LR: 8.583e-03
Train: 31 [  50/390]  Loss: 0.7216 (0.620)  Acc@1: 79.6875 (81.0355)  Acc@5: 98.4375 (97.5490)LR: 8.583e-03
Train: 31 [ 100/390]  Loss: 0.7549 (0.643)  Acc@1: 79.6875 (80.6776)  Acc@5: 93.7500 (97.2463)LR: 8.583e-03
Train: 31 [ 150/390]  Loss: 0.7546 (0.652)  Acc@1: 82.8125 (80.2152)  Acc@5: 93.7500 (97.1647)LR: 8.583e-03
Train: 31 [ 200/390]  Loss: 0.8070 (0.663)  Acc@1: 73.4375 (79.9052)  Acc@5: 98.4375 (97.0771)LR: 8.583e-03
Train: 31 [ 250/390]  Loss: 0.7007 (0.669)  Acc@1: 82.8125 (79.6875)  Acc@5: 96.8750 (96.9871)LR: 8.583e-03
Train: 31 [ 300/390]  Loss: 0.7100 (0.681)  Acc@1: 73.4375 (79.4124)  Acc@5: 98.4375 (96.7712)LR: 8.583e-03
Train: 31 [ 350/390]  Loss: 0.8567 (0.688)  Acc@1: 68.7500 (79.1311)  Acc@5: 96.8750 (96.7325)LR: 8.583e-03
Train: 31 [ 390/390]  Loss: 0.6968 (0.692)  Acc@1: 72.5000 (79.0520)  Acc@5: 97.5000 (96.7040)LR: 8.583e-03
train_acc 79.052000
Valid: 31 [   0/390]  Loss: 1.315 (1.32)  Acc@1: 65.6250 (65.6250)  Acc@5: 84.3750 (84.3750)
Valid: 31 [  50/390]  Loss: 1.202 (1.65)  Acc@1: 67.1875 (58.2414)  Acc@5: 89.0625 (85.7843)
Valid: 31 [ 100/390]  Loss: 1.756 (1.65)  Acc@1: 59.3750 (58.5087)  Acc@5: 76.5625 (85.3960)
Valid: 31 [ 150/390]  Loss: 1.281 (1.67)  Acc@1: 60.9375 (58.0091)  Acc@5: 90.6250 (85.1097)
Valid: 31 [ 200/390]  Loss: 1.604 (1.68)  Acc@1: 57.8125 (57.8125)  Acc@5: 85.9375 (85.0824)
Valid: 31 [ 250/390]  Loss: 1.332 (1.68)  Acc@1: 64.0625 (57.9806)  Acc@5: 89.0625 (85.1158)
Valid: 31 [ 300/390]  Loss: 1.380 (1.67)  Acc@1: 60.9375 (58.0409)  Acc@5: 89.0625 (85.2211)
Valid: 31 [ 350/390]  Loss: 1.473 (1.66)  Acc@1: 59.3750 (58.1508)  Acc@5: 95.3125 (85.4389)
Valid: 31 [ 390/390]  Loss: 1.898 (1.65)  Acc@1: 57.5000 (58.3520)  Acc@5: 82.5000 (85.5560)
valid_acc 58.352000
epoch = 31   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1764, 0.8236],
        [0.2247, 0.7753],
        [0.2300, 0.7700],
        [0.3809, 0.6191],
        [0.2344, 0.7656],
        [0.3226, 0.6774],
        [0.4544, 0.5456],
        [0.2403, 0.7597],
        [0.1326, 0.8674],
        [0.2655, 0.7345],
        [0.4143, 0.5857],
        [0.3011, 0.6989],
        [0.1060, 0.8940],
        [0.0617, 0.9383]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3180, 0.6820],
        [0.2644, 0.7356],
        [0.2355, 0.7645],
        [0.3071, 0.6929],
        [0.2547, 0.7453],
        [0.3816, 0.6184],
        [0.2253, 0.7747],
        [0.2623, 0.7377],
        [0.3099, 0.6901],
        [0.4227, 0.5773],
        [0.3311, 0.6689],
        [0.2747, 0.7253],
        [0.3566, 0.6434],
        [0.3002, 0.6998]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 32 [   0/390]  Loss: 0.6751 (0.675)  Acc@1: 75.0000 (75.0000)  Acc@5: 98.4375 (98.4375)LR: 7.891e-03
Train: 32 [  50/390]  Loss: 0.7736 (0.621)  Acc@1: 78.1250 (81.0355)  Acc@5: 93.7500 (97.3652)LR: 7.891e-03
Train: 32 [ 100/390]  Loss: 0.8654 (0.637)  Acc@1: 70.3125 (80.4146)  Acc@5: 95.3125 (97.2618)LR: 7.891e-03
Train: 32 [ 150/390]  Loss: 0.6690 (0.649)  Acc@1: 81.2500 (80.0083)  Acc@5: 96.8750 (97.2268)LR: 7.891e-03
Train: 32 [ 200/390]  Loss: 0.7021 (0.648)  Acc@1: 81.2500 (80.2627)  Acc@5: 93.7500 (97.1626)LR: 7.891e-03
Train: 32 [ 250/390]  Loss: 0.7769 (0.644)  Acc@1: 78.1250 (80.2229)  Acc@5: 96.8750 (97.2485)LR: 7.891e-03
Train: 32 [ 300/390]  Loss: 0.6745 (0.652)  Acc@1: 78.1250 (79.9522)  Acc@5: 96.8750 (97.1968)LR: 7.891e-03
Train: 32 [ 350/390]  Loss: 0.7481 (0.654)  Acc@1: 71.8750 (79.7721)  Acc@5: 100.0000 (97.2445)LR: 7.891e-03
Train: 32 [ 390/390]  Loss: 0.6719 (0.663)  Acc@1: 82.5000 (79.5560)  Acc@5: 97.5000 (97.0920)LR: 7.891e-03
train_acc 79.556000
Valid: 32 [   0/390]  Loss: 1.568 (1.57)  Acc@1: 59.3750 (59.3750)  Acc@5: 84.3750 (84.3750)
Valid: 32 [  50/390]  Loss: 1.534 (1.63)  Acc@1: 64.0625 (59.3750)  Acc@5: 85.9375 (85.7537)
Valid: 32 [ 100/390]  Loss: 1.555 (1.62)  Acc@1: 62.5000 (59.3750)  Acc@5: 89.0625 (85.8447)
Valid: 32 [ 150/390]  Loss: 1.771 (1.63)  Acc@1: 54.6875 (59.4060)  Acc@5: 82.8125 (85.6788)
Valid: 32 [ 200/390]  Loss: 1.537 (1.64)  Acc@1: 50.0000 (59.3206)  Acc@5: 89.0625 (85.7432)
Valid: 32 [ 250/390]  Loss: 1.633 (1.63)  Acc@1: 60.9375 (59.4684)  Acc@5: 89.0625 (85.8005)
Valid: 32 [ 300/390]  Loss: 1.535 (1.62)  Acc@1: 57.8125 (59.3542)  Acc@5: 84.3750 (85.9167)
Valid: 32 [ 350/390]  Loss: 1.659 (1.61)  Acc@1: 59.3750 (59.5308)  Acc@5: 82.8125 (85.8885)
Valid: 32 [ 390/390]  Loss: 1.478 (1.62)  Acc@1: 52.5000 (59.4840)  Acc@5: 90.0000 (85.8960)
valid_acc 59.484000
epoch = 32   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1725, 0.8275],
        [0.2219, 0.7781],
        [0.2299, 0.7701],
        [0.3793, 0.6207],
        [0.2352, 0.7648],
        [0.3220, 0.6780],
        [0.4555, 0.5445],
        [0.2439, 0.7561],
        [0.1283, 0.8717],
        [0.2631, 0.7369],
        [0.4191, 0.5809],
        [0.3052, 0.6948],
        [0.1012, 0.8988],
        [0.0579, 0.9421]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3162, 0.6838],
        [0.2569, 0.7431],
        [0.2291, 0.7709],
        [0.2988, 0.7012],
        [0.2483, 0.7517],
        [0.3801, 0.6199],
        [0.2174, 0.7826],
        [0.2568, 0.7432],
        [0.3061, 0.6939],
        [0.4191, 0.5809],
        [0.3262, 0.6738],
        [0.2682, 0.7318],
        [0.3518, 0.6482],
        [0.2974, 0.7026]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 33 [   0/390]  Loss: 0.6782 (0.678)  Acc@1: 79.6875 (79.6875)  Acc@5: 96.8750 (96.8750)LR: 7.219e-03
Train: 33 [  50/390]  Loss: 0.8609 (0.594)  Acc@1: 76.5625 (81.5564)  Acc@5: 92.1875 (97.6103)LR: 7.219e-03
Train: 33 [ 100/390]  Loss: 0.3364 (0.583)  Acc@1: 85.9375 (81.9616)  Acc@5: 98.4375 (97.8032)LR: 7.219e-03
Train: 33 [ 150/390]  Loss: 0.5790 (0.590)  Acc@1: 79.6875 (81.7053)  Acc@5: 98.4375 (97.7339)LR: 7.219e-03
Train: 33 [ 200/390]  Loss: 0.6670 (0.593)  Acc@1: 79.6875 (81.6620)  Acc@5: 98.4375 (97.6679)LR: 7.219e-03
Train: 33 [ 250/390]  Loss: 0.8028 (0.601)  Acc@1: 73.4375 (81.3434)  Acc@5: 98.4375 (97.5473)LR: 7.219e-03
Train: 33 [ 300/390]  Loss: 0.6644 (0.603)  Acc@1: 84.3750 (81.2240)  Acc@5: 98.4375 (97.5550)LR: 7.219e-03
Train: 33 [ 350/390]  Loss: 0.7467 (0.608)  Acc@1: 75.0000 (81.0452)  Acc@5: 100.0000 (97.5249)LR: 7.219e-03
Train: 33 [ 390/390]  Loss: 0.4869 (0.615)  Acc@1: 77.5000 (80.8920)  Acc@5: 100.0000 (97.4120)LR: 7.219e-03
train_acc 80.892000
Valid: 33 [   0/390]  Loss: 1.377 (1.38)  Acc@1: 60.9375 (60.9375)  Acc@5: 90.6250 (90.6250)
Valid: 33 [  50/390]  Loss: 1.698 (1.67)  Acc@1: 56.2500 (58.4865)  Acc@5: 82.8125 (84.6201)
Valid: 33 [ 100/390]  Loss: 1.702 (1.68)  Acc@1: 54.6875 (58.0446)  Acc@5: 87.5000 (85.3806)
Valid: 33 [ 150/390]  Loss: 1.265 (1.66)  Acc@1: 60.9375 (58.8990)  Acc@5: 89.0625 (85.6478)
Valid: 33 [ 200/390]  Loss: 1.515 (1.66)  Acc@1: 65.6250 (58.5665)  Acc@5: 90.6250 (85.6654)
Valid: 33 [ 250/390]  Loss: 1.563 (1.65)  Acc@1: 70.3125 (58.9206)  Acc@5: 85.9375 (85.8628)
Valid: 33 [ 300/390]  Loss: 1.948 (1.64)  Acc@1: 56.2500 (59.0635)  Acc@5: 90.6250 (85.9064)
Valid: 33 [ 350/390]  Loss: 1.476 (1.64)  Acc@1: 60.9375 (59.0901)  Acc@5: 85.9375 (86.0176)
Valid: 33 [ 390/390]  Loss: 1.267 (1.63)  Acc@1: 67.5000 (59.2480)  Acc@5: 92.5000 (86.0400)
valid_acc 59.248000
epoch = 33   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1656, 0.8344],
        [0.2171, 0.7829],
        [0.2236, 0.7764],
        [0.3792, 0.6208],
        [0.2350, 0.7650],
        [0.3201, 0.6799],
        [0.4613, 0.5387],
        [0.2470, 0.7530],
        [0.1261, 0.8739],
        [0.2610, 0.7390],
        [0.4301, 0.5699],
        [0.3093, 0.6907],
        [0.0972, 0.9028],
        [0.0543, 0.9457]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3101, 0.6899],
        [0.2516, 0.7484],
        [0.2232, 0.7768],
        [0.2952, 0.7048],
        [0.2433, 0.7567],
        [0.3805, 0.6195],
        [0.2102, 0.7898],
        [0.2525, 0.7475],
        [0.3055, 0.6945],
        [0.4166, 0.5834],
        [0.3250, 0.6750],
        [0.2630, 0.7370],
        [0.3461, 0.6539],
        [0.2924, 0.7076]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 34 [   0/390]  Loss: 0.5472 (0.547)  Acc@1: 84.3750 (84.3750)  Acc@5: 96.8750 (96.8750)LR: 6.570e-03
Train: 34 [  50/390]  Loss: 0.5309 (0.539)  Acc@1: 81.2500 (83.6397)  Acc@5: 98.4375 (98.1618)LR: 6.570e-03
Train: 34 [ 100/390]  Loss: 0.4846 (0.527)  Acc@1: 84.3750 (83.9728)  Acc@5: 98.4375 (98.2364)LR: 6.570e-03
Train: 34 [ 150/390]  Loss: 0.5936 (0.536)  Acc@1: 81.2500 (83.5161)  Acc@5: 98.4375 (98.1892)LR: 6.570e-03
Train: 34 [ 200/390]  Loss: 0.6083 (0.542)  Acc@1: 82.8125 (83.4499)  Acc@5: 96.8750 (98.1110)LR: 6.570e-03
Train: 34 [ 250/390]  Loss: 0.5572 (0.546)  Acc@1: 82.8125 (83.3167)  Acc@5: 98.4375 (98.0764)LR: 6.570e-03
Train: 34 [ 300/390]  Loss: 0.5075 (0.552)  Acc@1: 82.8125 (83.0409)  Acc@5: 98.4375 (97.9963)LR: 6.570e-03
Train: 34 [ 350/390]  Loss: 0.4028 (0.559)  Acc@1: 82.8125 (82.7101)  Acc@5: 100.0000 (97.9434)LR: 6.570e-03
Train: 34 [ 390/390]  Loss: 1.058 (0.565)  Acc@1: 67.5000 (82.5480)  Acc@5: 95.0000 (97.8760)LR: 6.570e-03
train_acc 82.548000
Valid: 34 [   0/390]  Loss: 1.573 (1.57)  Acc@1: 57.8125 (57.8125)  Acc@5: 85.9375 (85.9375)
Valid: 34 [  50/390]  Loss: 1.666 (1.65)  Acc@1: 65.6250 (59.3444)  Acc@5: 82.8125 (86.3358)
Valid: 34 [ 100/390]  Loss: 2.216 (1.66)  Acc@1: 50.0000 (58.6170)  Acc@5: 78.1250 (86.1231)
Valid: 34 [ 150/390]  Loss: 1.675 (1.68)  Acc@1: 62.5000 (58.8887)  Acc@5: 84.3750 (85.8133)
Valid: 34 [ 200/390]  Loss: 1.855 (1.68)  Acc@1: 50.0000 (58.8386)  Acc@5: 85.9375 (85.8053)
Valid: 34 [ 250/390]  Loss: 1.931 (1.67)  Acc@1: 62.5000 (58.9579)  Acc@5: 82.8125 (85.9375)
Valid: 34 [ 300/390]  Loss: 2.125 (1.67)  Acc@1: 39.0625 (59.1414)  Acc@5: 81.2500 (86.0257)
Valid: 34 [ 350/390]  Loss: 1.405 (1.66)  Acc@1: 60.9375 (59.3572)  Acc@5: 90.6250 (86.1334)
Valid: 34 [ 390/390]  Loss: 2.192 (1.66)  Acc@1: 45.0000 (59.3720)  Acc@5: 82.5000 (86.1400)
valid_acc 59.372000
epoch = 34   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1593, 0.8407],
        [0.2144, 0.7856],
        [0.2222, 0.7778],
        [0.3827, 0.6173],
        [0.2344, 0.7656],
        [0.3178, 0.6822],
        [0.4663, 0.5337],
        [0.2509, 0.7491],
        [0.1244, 0.8756],
        [0.2547, 0.7453],
        [0.4392, 0.5608],
        [0.3116, 0.6884],
        [0.0944, 0.9056],
        [0.0504, 0.9496]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3039, 0.6961],
        [0.2473, 0.7527],
        [0.2178, 0.7822],
        [0.2884, 0.7116],
        [0.2398, 0.7602],
        [0.3792, 0.6208],
        [0.2073, 0.7927],
        [0.2473, 0.7527],
        [0.3010, 0.6990],
        [0.4123, 0.5877],
        [0.3220, 0.6780],
        [0.2604, 0.7396],
        [0.3442, 0.6558],
        [0.2869, 0.7131]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 35 [   0/390]  Loss: 0.4784 (0.478)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)LR: 5.947e-03
Train: 35 [  50/390]  Loss: 0.5086 (0.492)  Acc@1: 84.3750 (85.0797)  Acc@5: 96.8750 (98.5907)LR: 5.947e-03
Train: 35 [ 100/390]  Loss: 0.3592 (0.500)  Acc@1: 89.0625 (84.7463)  Acc@5: 98.4375 (98.3292)LR: 5.947e-03
Train: 35 [ 150/390]  Loss: 0.6015 (0.505)  Acc@1: 82.8125 (84.4785)  Acc@5: 96.8750 (98.2719)LR: 5.947e-03
Train: 35 [ 200/390]  Loss: 0.6240 (0.514)  Acc@1: 84.3750 (84.3517)  Acc@5: 96.8750 (98.1965)LR: 5.947e-03
Train: 35 [ 250/390]  Loss: 0.4645 (0.516)  Acc@1: 87.5000 (84.3625)  Acc@5: 98.4375 (98.2694)LR: 5.947e-03
Train: 35 [ 300/390]  Loss: 0.7109 (0.521)  Acc@1: 79.6875 (84.1622)  Acc@5: 98.4375 (98.1987)LR: 5.947e-03
Train: 35 [ 350/390]  Loss: 0.6376 (0.523)  Acc@1: 81.2500 (83.9877)  Acc@5: 95.3125 (98.1838)LR: 5.947e-03
Train: 35 [ 390/390]  Loss: 0.4514 (0.528)  Acc@1: 85.0000 (83.8280)  Acc@5: 100.0000 (98.1400)LR: 5.947e-03
train_acc 83.828000
Valid: 35 [   0/390]  Loss: 1.623 (1.62)  Acc@1: 62.5000 (62.5000)  Acc@5: 82.8125 (82.8125)
Valid: 35 [  50/390]  Loss: 1.506 (1.54)  Acc@1: 56.2500 (61.7341)  Acc@5: 89.0625 (87.0711)
Valid: 35 [ 100/390]  Loss: 1.674 (1.56)  Acc@1: 64.0625 (61.2005)  Acc@5: 87.5000 (86.9895)
Valid: 35 [ 150/390]  Loss: 1.725 (1.59)  Acc@1: 60.9375 (60.8754)  Acc@5: 85.9375 (86.6515)
Valid: 35 [ 200/390]  Loss: 1.742 (1.57)  Acc@1: 48.4375 (61.0852)  Acc@5: 84.3750 (86.7848)
Valid: 35 [ 250/390]  Loss: 2.599 (1.58)  Acc@1: 48.4375 (60.9437)  Acc@5: 78.1250 (86.8028)
Valid: 35 [ 300/390]  Loss: 1.269 (1.58)  Acc@1: 68.7500 (60.8700)  Acc@5: 89.0625 (86.8407)
Valid: 35 [ 350/390]  Loss: 1.771 (1.59)  Acc@1: 57.8125 (60.5992)  Acc@5: 82.8125 (86.8189)
Valid: 35 [ 390/390]  Loss: 2.000 (1.60)  Acc@1: 47.5000 (60.5040)  Acc@5: 80.0000 (86.6560)
valid_acc 60.504000
epoch = 35   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1528, 0.8472],
        [0.2118, 0.7882],
        [0.2217, 0.7783],
        [0.3848, 0.6152],
        [0.2349, 0.7651],
        [0.3152, 0.6848],
        [0.4724, 0.5276],
        [0.2544, 0.7456],
        [0.1217, 0.8783],
        [0.2513, 0.7487],
        [0.4416, 0.5584],
        [0.3109, 0.6891],
        [0.0909, 0.9091],
        [0.0478, 0.9522]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.3000, 0.7000],
        [0.2447, 0.7553],
        [0.2123, 0.7877],
        [0.2868, 0.7132],
        [0.2355, 0.7645],
        [0.3759, 0.6241],
        [0.2044, 0.7956],
        [0.2434, 0.7566],
        [0.2963, 0.7037],
        [0.4123, 0.5877],
        [0.3185, 0.6815],
        [0.2545, 0.7455],
        [0.3396, 0.6604],
        [0.2809, 0.7191]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 36 [   0/390]  Loss: 0.3839 (0.384)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)LR: 5.351e-03
Train: 36 [  50/390]  Loss: 0.4819 (0.464)  Acc@1: 85.9375 (86.4890)  Acc@5: 96.8750 (98.7439)LR: 5.351e-03
Train: 36 [ 100/390]  Loss: 0.3639 (0.462)  Acc@1: 85.9375 (86.3861)  Acc@5: 100.0000 (98.7624)LR: 5.351e-03
Train: 36 [ 150/390]  Loss: 0.4112 (0.456)  Acc@1: 87.5000 (86.3618)  Acc@5: 98.4375 (98.7686)LR: 5.351e-03
Train: 36 [ 200/390]  Loss: 0.3669 (0.467)  Acc@1: 84.3750 (85.7587)  Acc@5: 100.0000 (98.7640)LR: 5.351e-03
Train: 36 [ 250/390]  Loss: 0.4951 (0.471)  Acc@1: 81.2500 (85.6885)  Acc@5: 98.4375 (98.7239)LR: 5.351e-03
Train: 36 [ 300/390]  Loss: 0.4517 (0.476)  Acc@1: 90.6250 (85.4703)  Acc@5: 96.8750 (98.6711)LR: 5.351e-03
Train: 36 [ 350/390]  Loss: 0.6447 (0.480)  Acc@1: 75.0000 (85.2920)  Acc@5: 100.0000 (98.6334)LR: 5.351e-03
Train: 36 [ 390/390]  Loss: 0.3851 (0.486)  Acc@1: 90.0000 (85.0840)  Acc@5: 97.5000 (98.5600)LR: 5.351e-03
train_acc 85.084000
Valid: 36 [   0/390]  Loss: 1.764 (1.76)  Acc@1: 59.3750 (59.3750)  Acc@5: 84.3750 (84.3750)
Valid: 36 [  50/390]  Loss: 2.033 (1.65)  Acc@1: 53.1250 (59.5282)  Acc@5: 82.8125 (86.6728)
Valid: 36 [ 100/390]  Loss: 1.645 (1.71)  Acc@1: 56.2500 (59.5606)  Acc@5: 84.3750 (85.6745)
Valid: 36 [ 150/390]  Loss: 1.930 (1.70)  Acc@1: 54.6875 (59.3957)  Acc@5: 85.9375 (85.7305)
Valid: 36 [ 200/390]  Loss: 1.130 (1.69)  Acc@1: 70.3125 (59.5927)  Acc@5: 90.6250 (85.8442)
Valid: 36 [ 250/390]  Loss: 1.417 (1.69)  Acc@1: 60.9375 (59.3439)  Acc@5: 85.9375 (85.9935)
Valid: 36 [ 300/390]  Loss: 1.519 (1.67)  Acc@1: 65.6250 (59.6138)  Acc@5: 87.5000 (86.1400)
Valid: 36 [ 350/390]  Loss: 1.291 (1.66)  Acc@1: 70.3125 (59.6911)  Acc@5: 85.9375 (86.2046)
Valid: 36 [ 390/390]  Loss: 1.400 (1.66)  Acc@1: 62.5000 (59.8360)  Acc@5: 90.0000 (86.2760)
valid_acc 59.836000
epoch = 36   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1463, 0.8537],
        [0.2076, 0.7924],
        [0.2234, 0.7766],
        [0.3845, 0.6155],
        [0.2392, 0.7608],
        [0.3161, 0.6839],
        [0.4749, 0.5251],
        [0.2546, 0.7454],
        [0.1185, 0.8815],
        [0.2511, 0.7489],
        [0.4486, 0.5514],
        [0.3094, 0.6906],
        [0.0881, 0.9119],
        [0.0451, 0.9549]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2961, 0.7039],
        [0.2408, 0.7592],
        [0.2093, 0.7907],
        [0.2806, 0.7194],
        [0.2305, 0.7695],
        [0.3784, 0.6216],
        [0.1996, 0.8004],
        [0.2415, 0.7585],
        [0.2944, 0.7056],
        [0.4147, 0.5853],
        [0.3127, 0.6873],
        [0.2486, 0.7514],
        [0.3405, 0.6595],
        [0.2790, 0.7210]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 37 [   0/390]  Loss: 0.4479 (0.448)  Acc@1: 87.5000 (87.5000)  Acc@5: 96.8750 (96.8750)LR: 4.785e-03
Train: 37 [  50/390]  Loss: 0.5316 (0.437)  Acc@1: 81.2500 (86.6728)  Acc@5: 100.0000 (98.4681)LR: 4.785e-03
Train: 37 [ 100/390]  Loss: 0.4180 (0.446)  Acc@1: 82.8125 (86.1386)  Acc@5: 100.0000 (98.6541)LR: 4.785e-03
Train: 37 [ 150/390]  Loss: 0.4209 (0.453)  Acc@1: 87.5000 (85.6995)  Acc@5: 98.4375 (98.6651)LR: 4.785e-03
Train: 37 [ 200/390]  Loss: 0.5877 (0.450)  Acc@1: 87.5000 (85.9764)  Acc@5: 95.3125 (98.6552)LR: 4.785e-03
Train: 37 [ 250/390]  Loss: 0.4293 (0.446)  Acc@1: 92.1875 (86.1803)  Acc@5: 100.0000 (98.6990)LR: 4.785e-03
Train: 37 [ 300/390]  Loss: 0.4243 (0.449)  Acc@1: 85.9375 (86.1555)  Acc@5: 98.4375 (98.6607)LR: 4.785e-03
Train: 37 [ 350/390]  Loss: 0.4674 (0.453)  Acc@1: 89.0625 (85.8885)  Acc@5: 98.4375 (98.6467)LR: 4.785e-03
Train: 37 [ 390/390]  Loss: 0.7701 (0.456)  Acc@1: 72.5000 (85.7880)  Acc@5: 97.5000 (98.6480)LR: 4.785e-03
train_acc 85.788000
Valid: 37 [   0/390]  Loss: 1.208 (1.21)  Acc@1: 65.6250 (65.6250)  Acc@5: 90.6250 (90.6250)
Valid: 37 [  50/390]  Loss: 1.412 (1.61)  Acc@1: 68.7500 (61.6115)  Acc@5: 92.1875 (86.8873)
Valid: 37 [ 100/390]  Loss: 1.552 (1.64)  Acc@1: 64.0625 (60.9066)  Acc@5: 84.3750 (86.3243)
Valid: 37 [ 150/390]  Loss: 1.676 (1.64)  Acc@1: 59.3750 (60.9582)  Acc@5: 84.3750 (86.4342)
Valid: 37 [ 200/390]  Loss: 1.741 (1.62)  Acc@1: 57.8125 (61.0463)  Acc@5: 85.9375 (86.6838)
Valid: 37 [ 250/390]  Loss: 1.291 (1.62)  Acc@1: 64.0625 (60.9437)  Acc@5: 92.1875 (86.8090)
Valid: 37 [ 300/390]  Loss: 1.558 (1.62)  Acc@1: 68.7500 (60.9064)  Acc@5: 87.5000 (86.7681)
Valid: 37 [ 350/390]  Loss: 1.627 (1.62)  Acc@1: 67.1875 (60.8707)  Acc@5: 84.3750 (86.8278)
Valid: 37 [ 390/390]  Loss: 1.639 (1.62)  Acc@1: 57.5000 (60.9360)  Acc@5: 92.5000 (86.8680)
valid_acc 60.936000
epoch = 37   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1405, 0.8595],
        [0.2023, 0.7977],
        [0.2204, 0.7796],
        [0.3868, 0.6132],
        [0.2388, 0.7612],
        [0.3161, 0.6839],
        [0.4758, 0.5242],
        [0.2603, 0.7397],
        [0.1172, 0.8828],
        [0.2502, 0.7498],
        [0.4561, 0.5439],
        [0.3168, 0.6832],
        [0.0877, 0.9123],
        [0.0429, 0.9571]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2908, 0.7092],
        [0.2416, 0.7584],
        [0.2028, 0.7972],
        [0.2787, 0.7213],
        [0.2247, 0.7753],
        [0.3763, 0.6237],
        [0.1949, 0.8051],
        [0.2372, 0.7628],
        [0.2891, 0.7109],
        [0.4110, 0.5890],
        [0.3148, 0.6852],
        [0.2435, 0.7565],
        [0.3381, 0.6619],
        [0.2769, 0.7231]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 38 [   0/390]  Loss: 0.3718 (0.372)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)LR: 4.252e-03
Train: 38 [  50/390]  Loss: 0.3211 (0.394)  Acc@1: 93.7500 (88.4804)  Acc@5: 100.0000 (99.0196)LR: 4.252e-03
Train: 38 [ 100/390]  Loss: 0.3866 (0.406)  Acc@1: 85.9375 (87.5464)  Acc@5: 96.8750 (98.9480)LR: 4.252e-03
Train: 38 [ 150/390]  Loss: 0.3646 (0.407)  Acc@1: 89.0625 (87.5828)  Acc@5: 100.0000 (98.9756)LR: 4.252e-03
Train: 38 [ 200/390]  Loss: 0.5101 (0.410)  Acc@1: 84.3750 (87.4456)  Acc@5: 96.8750 (98.9894)LR: 4.252e-03
Train: 38 [ 250/390]  Loss: 0.4083 (0.414)  Acc@1: 87.5000 (87.3568)  Acc@5: 100.0000 (98.9729)LR: 4.252e-03
Train: 38 [ 300/390]  Loss: 0.4563 (0.413)  Acc@1: 84.3750 (87.3910)  Acc@5: 100.0000 (98.9877)LR: 4.252e-03
Train: 38 [ 350/390]  Loss: 0.4107 (0.417)  Acc@1: 89.0625 (87.2151)  Acc@5: 98.4375 (98.9717)LR: 4.252e-03
Train: 38 [ 390/390]  Loss: 0.7060 (0.418)  Acc@1: 75.0000 (87.1080)  Acc@5: 97.5000 (98.9880)LR: 4.252e-03
train_acc 87.108000
Valid: 38 [   0/390]  Loss: 1.663 (1.66)  Acc@1: 53.1250 (53.1250)  Acc@5: 85.9375 (85.9375)
Valid: 38 [  50/390]  Loss: 1.478 (1.65)  Acc@1: 65.6250 (59.7120)  Acc@5: 87.5000 (86.7341)
Valid: 38 [ 100/390]  Loss: 2.220 (1.61)  Acc@1: 56.2500 (60.6126)  Acc@5: 84.3750 (87.1442)
Valid: 38 [ 150/390]  Loss: 1.567 (1.62)  Acc@1: 59.3750 (60.3787)  Acc@5: 90.6250 (87.0447)
Valid: 38 [ 200/390]  Loss: 1.785 (1.63)  Acc@1: 59.3750 (60.7121)  Acc@5: 84.3750 (86.8703)
Valid: 38 [ 250/390]  Loss: 1.816 (1.63)  Acc@1: 56.2500 (60.6698)  Acc@5: 84.3750 (86.8152)
Valid: 38 [ 300/390]  Loss: 1.245 (1.62)  Acc@1: 62.5000 (60.8596)  Acc@5: 90.6250 (86.8719)
Valid: 38 [ 350/390]  Loss: 1.261 (1.62)  Acc@1: 64.0625 (60.9330)  Acc@5: 96.8750 (86.8768)
Valid: 38 [ 390/390]  Loss: 1.540 (1.62)  Acc@1: 67.5000 (60.9080)  Acc@5: 90.0000 (86.9440)
valid_acc 60.908000
epoch = 38   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1356, 0.8644],
        [0.1991, 0.8009],
        [0.2150, 0.7850],
        [0.3935, 0.6065],
        [0.2407, 0.7593],
        [0.3162, 0.6838],
        [0.4796, 0.5204],
        [0.2660, 0.7340],
        [0.1163, 0.8837],
        [0.2528, 0.7472],
        [0.4656, 0.5344],
        [0.3203, 0.6797],
        [0.0859, 0.9141],
        [0.0411, 0.9589]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2874, 0.7126],
        [0.2406, 0.7594],
        [0.1990, 0.8010],
        [0.2724, 0.7276],
        [0.2230, 0.7770],
        [0.3720, 0.6280],
        [0.1898, 0.8102],
        [0.2361, 0.7639],
        [0.2839, 0.7161],
        [0.4099, 0.5901],
        [0.3154, 0.6846],
        [0.2406, 0.7594],
        [0.3327, 0.6673],
        [0.2762, 0.7238]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 39 [   0/390]  Loss: 0.4728 (0.473)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [  50/390]  Loss: 0.3239 (0.379)  Acc@1: 92.1875 (88.6949)  Acc@5: 98.4375 (99.2341)LR: 3.754e-03
Train: 39 [ 100/390]  Loss: 0.5313 (0.375)  Acc@1: 79.6875 (88.8923)  Acc@5: 98.4375 (99.3812)LR: 3.754e-03
Train: 39 [ 150/390]  Loss: 0.3694 (0.375)  Acc@1: 85.9375 (88.8659)  Acc@5: 100.0000 (99.2446)LR: 3.754e-03
Train: 39 [ 200/390]  Loss: 0.4714 (0.379)  Acc@1: 82.8125 (88.6660)  Acc@5: 98.4375 (99.1449)LR: 3.754e-03
Train: 39 [ 250/390]  Loss: 0.3728 (0.384)  Acc@1: 90.6250 (88.5085)  Acc@5: 98.4375 (99.0725)LR: 3.754e-03
Train: 39 [ 300/390]  Loss: 0.5384 (0.386)  Acc@1: 87.5000 (88.3461)  Acc@5: 96.8750 (99.0604)LR: 3.754e-03
Train: 39 [ 350/390]  Loss: 0.1529 (0.383)  Acc@1: 98.4375 (88.4081)  Acc@5: 100.0000 (99.0830)LR: 3.754e-03
Train: 39 [ 390/390]  Loss: 0.3420 (0.385)  Acc@1: 92.5000 (88.2440)  Acc@5: 100.0000 (99.0960)LR: 3.754e-03
train_acc 88.244000
Valid: 39 [   0/390]  Loss: 1.593 (1.59)  Acc@1: 64.0625 (64.0625)  Acc@5: 85.9375 (85.9375)
Valid: 39 [  50/390]  Loss: 1.346 (1.62)  Acc@1: 62.5000 (60.0490)  Acc@5: 92.1875 (86.7953)
Valid: 39 [ 100/390]  Loss: 1.901 (1.60)  Acc@1: 59.3750 (60.9220)  Acc@5: 84.3750 (87.0978)
Valid: 39 [ 150/390]  Loss: 1.544 (1.62)  Acc@1: 65.6250 (60.9168)  Acc@5: 84.3750 (86.8067)
Valid: 39 [ 200/390]  Loss: 1.563 (1.61)  Acc@1: 62.5000 (61.1318)  Acc@5: 90.6250 (86.8626)
Valid: 39 [ 250/390]  Loss: 2.206 (1.61)  Acc@1: 48.4375 (61.1056)  Acc@5: 84.3750 (87.0518)
Valid: 39 [ 300/390]  Loss: 1.537 (1.60)  Acc@1: 64.0625 (61.1140)  Acc@5: 90.6250 (87.1262)
Valid: 39 [ 350/390]  Loss: 1.607 (1.60)  Acc@1: 68.7500 (61.1334)  Acc@5: 82.8125 (87.1617)
Valid: 39 [ 390/390]  Loss: 1.250 (1.60)  Acc@1: 67.5000 (61.1960)  Acc@5: 90.0000 (87.0680)
valid_acc 61.196000
epoch = 39   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1314, 0.8686],
        [0.1940, 0.8060],
        [0.2121, 0.7879],
        [0.3933, 0.6067],
        [0.2412, 0.7588],
        [0.3151, 0.6849],
        [0.4856, 0.5144],
        [0.2698, 0.7302],
        [0.1128, 0.8872],
        [0.2535, 0.7465],
        [0.4705, 0.5295],
        [0.3198, 0.6802],
        [0.0844, 0.9156],
        [0.0389, 0.9611]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2857, 0.7143],
        [0.2376, 0.7624],
        [0.1990, 0.8010],
        [0.2692, 0.7308],
        [0.2185, 0.7815],
        [0.3679, 0.6321],
        [0.1853, 0.8147],
        [0.2326, 0.7674],
        [0.2823, 0.7177],
        [0.4094, 0.5906],
        [0.3142, 0.6858],
        [0.2348, 0.7652],
        [0.3283, 0.6717],
        [0.2770, 0.7230]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 40 [   0/390]  Loss: 0.2606 (0.261)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 3.292e-03
Train: 40 [  50/390]  Loss: 0.4760 (0.337)  Acc@1: 84.3750 (90.6556)  Acc@5: 98.4375 (99.1115)LR: 3.292e-03
Train: 40 [ 100/390]  Loss: 0.3179 (0.348)  Acc@1: 87.5000 (89.8824)  Acc@5: 100.0000 (99.2110)LR: 3.292e-03
Train: 40 [ 150/390]  Loss: 0.3372 (0.343)  Acc@1: 95.3125 (89.9731)  Acc@5: 98.4375 (99.2653)LR: 3.292e-03
Train: 40 [ 200/390]  Loss: 0.2824 (0.353)  Acc@1: 90.6250 (89.6300)  Acc@5: 100.0000 (99.1682)LR: 3.292e-03
Train: 40 [ 250/390]  Loss: 0.3372 (0.353)  Acc@1: 92.1875 (89.5418)  Acc@5: 98.4375 (99.1596)LR: 3.292e-03
Train: 40 [ 300/390]  Loss: 0.3660 (0.356)  Acc@1: 89.0625 (89.4155)  Acc@5: 98.4375 (99.1642)LR: 3.292e-03
Train: 40 [ 350/390]  Loss: 0.2409 (0.361)  Acc@1: 95.3125 (89.2495)  Acc@5: 100.0000 (99.1275)LR: 3.292e-03
Train: 40 [ 390/390]  Loss: 0.4060 (0.361)  Acc@1: 92.5000 (89.2160)  Acc@5: 97.5000 (99.1360)LR: 3.292e-03
train_acc 89.216000
Valid: 40 [   0/390]  Loss: 1.233 (1.23)  Acc@1: 67.1875 (67.1875)  Acc@5: 90.6250 (90.6250)
Valid: 40 [  50/390]  Loss: 1.289 (1.56)  Acc@1: 62.5000 (62.6532)  Acc@5: 92.1875 (87.9289)
Valid: 40 [ 100/390]  Loss: 1.329 (1.57)  Acc@1: 67.1875 (62.3453)  Acc@5: 89.0625 (87.4072)
Valid: 40 [ 150/390]  Loss: 2.028 (1.58)  Acc@1: 60.9375 (62.2310)  Acc@5: 84.3750 (87.2206)
Valid: 40 [ 200/390]  Loss: 0.8996 (1.58)  Acc@1: 73.4375 (61.8237)  Acc@5: 93.7500 (87.3989)
Valid: 40 [ 250/390]  Loss: 1.206 (1.59)  Acc@1: 59.3750 (61.4729)  Acc@5: 90.6250 (87.3568)
Valid: 40 [ 300/390]  Loss: 1.868 (1.60)  Acc@1: 60.9375 (61.4826)  Acc@5: 81.2500 (87.1989)
Valid: 40 [ 350/390]  Loss: 2.008 (1.60)  Acc@1: 54.6875 (61.4806)  Acc@5: 84.3750 (87.2017)
Valid: 40 [ 390/390]  Loss: 1.999 (1.61)  Acc@1: 57.5000 (61.3360)  Acc@5: 85.0000 (87.1720)
valid_acc 61.336000
epoch = 40   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1271, 0.8729],
        [0.1930, 0.8070],
        [0.2121, 0.7879],
        [0.3918, 0.6082],
        [0.2431, 0.7569],
        [0.3127, 0.6873],
        [0.4867, 0.5133],
        [0.2731, 0.7269],
        [0.1101, 0.8899],
        [0.2561, 0.7439],
        [0.4768, 0.5232],
        [0.3251, 0.6749],
        [0.0820, 0.9180],
        [0.0374, 0.9626]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2838, 0.7162],
        [0.2327, 0.7673],
        [0.1956, 0.8044],
        [0.2656, 0.7344],
        [0.2136, 0.7864],
        [0.3674, 0.6326],
        [0.1799, 0.8201],
        [0.2279, 0.7721],
        [0.2841, 0.7159],
        [0.4114, 0.5886],
        [0.3105, 0.6895],
        [0.2285, 0.7715],
        [0.3258, 0.6742],
        [0.2724, 0.7276]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 41 [   0/390]  Loss: 0.3524 (0.352)  Acc@1: 92.1875 (92.1875)  Acc@5: 96.8750 (96.8750)LR: 2.868e-03
Train: 41 [  50/390]  Loss: 0.3234 (0.323)  Acc@1: 90.6250 (90.2574)  Acc@5: 100.0000 (99.5711)LR: 2.868e-03
Train: 41 [ 100/390]  Loss: 0.3138 (0.318)  Acc@1: 92.1875 (90.6095)  Acc@5: 98.4375 (99.4276)LR: 2.868e-03
Train: 41 [ 150/390]  Loss: 0.4117 (0.314)  Acc@1: 87.5000 (90.8630)  Acc@5: 100.0000 (99.4826)LR: 2.868e-03
Train: 41 [ 200/390]  Loss: 0.3901 (0.316)  Acc@1: 87.5000 (90.8815)  Acc@5: 100.0000 (99.5025)LR: 2.868e-03
Train: 41 [ 250/390]  Loss: 0.3661 (0.324)  Acc@1: 89.0625 (90.5441)  Acc@5: 98.4375 (99.4211)LR: 2.868e-03
Train: 41 [ 300/390]  Loss: 0.3053 (0.326)  Acc@1: 92.1875 (90.4848)  Acc@5: 100.0000 (99.3667)LR: 2.868e-03
Train: 41 [ 350/390]  Loss: 0.3083 (0.329)  Acc@1: 92.1875 (90.2956)  Acc@5: 100.0000 (99.3634)LR: 2.868e-03
Train: 41 [ 390/390]  Loss: 0.3968 (0.333)  Acc@1: 82.5000 (90.1080)  Acc@5: 100.0000 (99.3560)LR: 2.868e-03
train_acc 90.108000
Valid: 41 [   0/390]  Loss: 1.609 (1.61)  Acc@1: 60.9375 (60.9375)  Acc@5: 87.5000 (87.5000)
Valid: 41 [  50/390]  Loss: 2.033 (1.65)  Acc@1: 56.2500 (60.1716)  Acc@5: 82.8125 (86.9179)
Valid: 41 [ 100/390]  Loss: 1.592 (1.63)  Acc@1: 67.1875 (61.2005)  Acc@5: 85.9375 (86.8812)
Valid: 41 [ 150/390]  Loss: 1.967 (1.62)  Acc@1: 51.5625 (61.2065)  Acc@5: 82.8125 (87.0447)
Valid: 41 [ 200/390]  Loss: 1.569 (1.63)  Acc@1: 68.7500 (61.3029)  Acc@5: 90.6250 (87.1035)
Valid: 41 [ 250/390]  Loss: 1.587 (1.62)  Acc@1: 60.9375 (61.5227)  Acc@5: 84.3750 (87.3319)
Valid: 41 [ 300/390]  Loss: 1.509 (1.62)  Acc@1: 60.9375 (61.4877)  Acc@5: 92.1875 (87.3339)
Valid: 41 [ 350/390]  Loss: 1.630 (1.62)  Acc@1: 60.9375 (61.4895)  Acc@5: 81.2500 (87.3041)
Valid: 41 [ 390/390]  Loss: 1.635 (1.62)  Acc@1: 62.5000 (61.5120)  Acc@5: 85.0000 (87.2480)
valid_acc 61.512000
epoch = 41   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1218, 0.8782],
        [0.1925, 0.8075],
        [0.2126, 0.7874],
        [0.3921, 0.6079],
        [0.2462, 0.7538],
        [0.3111, 0.6889],
        [0.4906, 0.5094],
        [0.2753, 0.7247],
        [0.1075, 0.8925],
        [0.2571, 0.7429],
        [0.4816, 0.5184],
        [0.3266, 0.6734],
        [0.0808, 0.9192],
        [0.0362, 0.9638]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2832, 0.7168],
        [0.2296, 0.7704],
        [0.1924, 0.8076],
        [0.2601, 0.7399],
        [0.2121, 0.7879],
        [0.3660, 0.6340],
        [0.1749, 0.8251],
        [0.2251, 0.7749],
        [0.2824, 0.7176],
        [0.4114, 0.5886],
        [0.3094, 0.6906],
        [0.2273, 0.7727],
        [0.3284, 0.6716],
        [0.2714, 0.7286]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 42 [   0/390]  Loss: 0.2930 (0.293)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [  50/390]  Loss: 0.2143 (0.278)  Acc@1: 96.8750 (92.3713)  Acc@5: 100.0000 (99.6936)LR: 2.484e-03
Train: 42 [ 100/390]  Loss: 0.2745 (0.290)  Acc@1: 90.6250 (91.9709)  Acc@5: 100.0000 (99.5823)LR: 2.484e-03
Train: 42 [ 150/390]  Loss: 0.3011 (0.291)  Acc@1: 96.8750 (91.9392)  Acc@5: 98.4375 (99.5861)LR: 2.484e-03
Train: 42 [ 200/390]  Loss: 0.2950 (0.291)  Acc@1: 92.1875 (92.0009)  Acc@5: 100.0000 (99.5647)LR: 2.484e-03
Train: 42 [ 250/390]  Loss: 0.2070 (0.295)  Acc@1: 95.3125 (91.7704)  Acc@5: 100.0000 (99.5456)LR: 2.484e-03
Train: 42 [ 300/390]  Loss: 0.2798 (0.299)  Acc@1: 92.1875 (91.5750)  Acc@5: 100.0000 (99.4965)LR: 2.484e-03
Train: 42 [ 350/390]  Loss: 0.3365 (0.305)  Acc@1: 89.0625 (91.2304)  Acc@5: 98.4375 (99.4881)LR: 2.484e-03
Train: 42 [ 390/390]  Loss: 0.4111 (0.311)  Acc@1: 90.0000 (91.0520)  Acc@5: 97.5000 (99.4200)LR: 2.484e-03
train_acc 91.052000
Valid: 42 [   0/390]  Loss: 1.485 (1.49)  Acc@1: 57.8125 (57.8125)  Acc@5: 87.5000 (87.5000)
Valid: 42 [  50/390]  Loss: 1.101 (1.54)  Acc@1: 71.8750 (62.9289)  Acc@5: 90.6250 (88.0515)
Valid: 42 [ 100/390]  Loss: 1.605 (1.58)  Acc@1: 60.9375 (62.0050)  Acc@5: 85.9375 (87.5928)
Valid: 42 [ 150/390]  Loss: 1.033 (1.61)  Acc@1: 70.3125 (61.8481)  Acc@5: 93.7500 (87.2103)
Valid: 42 [ 200/390]  Loss: 1.643 (1.62)  Acc@1: 60.9375 (61.6993)  Acc@5: 89.0625 (87.2046)
Valid: 42 [ 250/390]  Loss: 1.814 (1.62)  Acc@1: 56.2500 (61.6036)  Acc@5: 84.3750 (87.1016)
Valid: 42 [ 300/390]  Loss: 1.646 (1.62)  Acc@1: 68.7500 (61.6902)  Acc@5: 85.9375 (87.1418)
Valid: 42 [ 350/390]  Loss: 1.471 (1.63)  Acc@1: 62.5000 (61.5118)  Acc@5: 87.5000 (87.1038)
Valid: 42 [ 390/390]  Loss: 1.978 (1.63)  Acc@1: 60.0000 (61.4360)  Acc@5: 82.5000 (87.0960)
valid_acc 61.436000
epoch = 42   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1162, 0.8838],
        [0.1914, 0.8086],
        [0.2116, 0.7884],
        [0.3878, 0.6122],
        [0.2473, 0.7527],
        [0.3099, 0.6901],
        [0.4957, 0.5043],
        [0.2771, 0.7229],
        [0.1065, 0.8935],
        [0.2561, 0.7439],
        [0.4825, 0.5175],
        [0.3288, 0.6712],
        [0.0791, 0.9209],
        [0.0348, 0.9652]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2793, 0.7207],
        [0.2237, 0.7763],
        [0.1888, 0.8112],
        [0.2574, 0.7426],
        [0.2091, 0.7909],
        [0.3608, 0.6392],
        [0.1691, 0.8309],
        [0.2244, 0.7756],
        [0.2808, 0.7192],
        [0.4151, 0.5849],
        [0.3024, 0.6976],
        [0.2236, 0.7764],
        [0.3229, 0.6771],
        [0.2667, 0.7333]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 43 [   0/390]  Loss: 0.4040 (0.404)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)LR: 2.142e-03
Train: 43 [  50/390]  Loss: 0.2864 (0.283)  Acc@1: 92.1875 (92.4326)  Acc@5: 100.0000 (99.5404)LR: 2.142e-03
Train: 43 [ 100/390]  Loss: 0.1562 (0.282)  Acc@1: 98.4375 (92.3113)  Acc@5: 100.0000 (99.6132)LR: 2.142e-03
Train: 43 [ 150/390]  Loss: 0.1629 (0.286)  Acc@1: 95.3125 (92.0012)  Acc@5: 100.0000 (99.5757)LR: 2.142e-03
Train: 43 [ 200/390]  Loss: 0.3352 (0.284)  Acc@1: 90.6250 (92.1564)  Acc@5: 100.0000 (99.5569)LR: 2.142e-03
Train: 43 [ 250/390]  Loss: 0.1995 (0.288)  Acc@1: 96.8750 (91.9634)  Acc@5: 100.0000 (99.5456)LR: 2.142e-03
Train: 43 [ 300/390]  Loss: 0.2667 (0.290)  Acc@1: 92.1875 (91.8968)  Acc@5: 100.0000 (99.5172)LR: 2.142e-03
Train: 43 [ 350/390]  Loss: 0.1468 (0.288)  Acc@1: 96.8750 (91.9071)  Acc@5: 100.0000 (99.5370)LR: 2.142e-03
Train: 43 [ 390/390]  Loss: 0.2461 (0.288)  Acc@1: 95.0000 (91.7920)  Acc@5: 100.0000 (99.5480)LR: 2.142e-03
train_acc 91.792000
Valid: 43 [   0/390]  Loss: 1.955 (1.96)  Acc@1: 56.2500 (56.2500)  Acc@5: 85.9375 (85.9375)
Valid: 43 [  50/390]  Loss: 1.885 (1.66)  Acc@1: 50.0000 (61.7953)  Acc@5: 87.5000 (87.1017)
Valid: 43 [ 100/390]  Loss: 1.395 (1.59)  Acc@1: 57.8125 (63.0105)  Acc@5: 90.6250 (87.7166)
Valid: 43 [ 150/390]  Loss: 1.527 (1.60)  Acc@1: 67.1875 (62.5724)  Acc@5: 92.1875 (87.4897)
Valid: 43 [ 200/390]  Loss: 1.065 (1.60)  Acc@1: 67.1875 (62.2901)  Acc@5: 93.7500 (87.4767)
Valid: 43 [ 250/390]  Loss: 1.559 (1.60)  Acc@1: 62.5000 (62.0829)  Acc@5: 89.0625 (87.4315)
Valid: 43 [ 300/390]  Loss: 1.459 (1.60)  Acc@1: 70.3125 (62.1470)  Acc@5: 95.3125 (87.5571)
Valid: 43 [ 350/390]  Loss: 1.334 (1.60)  Acc@1: 67.1875 (62.1572)  Acc@5: 90.6250 (87.4777)
Valid: 43 [ 390/390]  Loss: 1.960 (1.61)  Acc@1: 55.0000 (62.0800)  Acc@5: 85.0000 (87.4880)
valid_acc 62.080000
epoch = 43   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1115, 0.8885],
        [0.1891, 0.8109],
        [0.2104, 0.7896],
        [0.3897, 0.6103],
        [0.2467, 0.7533],
        [0.3086, 0.6914],
        [0.5008, 0.4992],
        [0.2820, 0.7180],
        [0.1053, 0.8947],
        [0.2526, 0.7474],
        [0.4844, 0.5156],
        [0.3281, 0.6719],
        [0.0779, 0.9221],
        [0.0333, 0.9667]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2754, 0.7246],
        [0.2188, 0.7812],
        [0.1853, 0.8147],
        [0.2531, 0.7469],
        [0.2072, 0.7928],
        [0.3560, 0.6440],
        [0.1645, 0.8355],
        [0.2239, 0.7761],
        [0.2773, 0.7227],
        [0.4154, 0.5846],
        [0.3016, 0.6984],
        [0.2193, 0.7807],
        [0.3213, 0.6787],
        [0.2641, 0.7359]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 44 [   0/390]  Loss: 0.3020 (0.302)  Acc@1: 89.0625 (89.0625)  Acc@5: 98.4375 (98.4375)LR: 1.843e-03
Train: 44 [  50/390]  Loss: 0.2446 (0.265)  Acc@1: 93.7500 (92.8002)  Acc@5: 100.0000 (99.7855)LR: 1.843e-03
Train: 44 [ 100/390]  Loss: 0.2595 (0.267)  Acc@1: 90.6250 (92.4041)  Acc@5: 100.0000 (99.6751)LR: 1.843e-03
Train: 44 [ 150/390]  Loss: 0.1337 (0.266)  Acc@1: 98.4375 (92.3841)  Acc@5: 100.0000 (99.6999)LR: 1.843e-03
Train: 44 [ 200/390]  Loss: 0.2342 (0.270)  Acc@1: 92.1875 (92.2341)  Acc@5: 100.0000 (99.6968)LR: 1.843e-03
Train: 44 [ 250/390]  Loss: 0.2210 (0.276)  Acc@1: 93.7500 (92.0692)  Acc@5: 100.0000 (99.6576)LR: 1.843e-03
Train: 44 [ 300/390]  Loss: 0.2116 (0.275)  Acc@1: 93.7500 (92.0733)  Acc@5: 100.0000 (99.6782)LR: 1.843e-03
Train: 44 [ 350/390]  Loss: 0.2461 (0.273)  Acc@1: 92.1875 (92.0673)  Acc@5: 100.0000 (99.6750)LR: 1.843e-03
Train: 44 [ 390/390]  Loss: 0.1610 (0.273)  Acc@1: 97.5000 (92.0560)  Acc@5: 100.0000 (99.6600)LR: 1.843e-03
train_acc 92.056000
Valid: 44 [   0/390]  Loss: 1.987 (1.99)  Acc@1: 59.3750 (59.3750)  Acc@5: 81.2500 (81.2500)
Valid: 44 [  50/390]  Loss: 1.427 (1.57)  Acc@1: 70.3125 (63.3885)  Acc@5: 84.3750 (87.7451)
Valid: 44 [ 100/390]  Loss: 1.375 (1.57)  Acc@1: 57.8125 (62.9022)  Acc@5: 89.0625 (88.1033)
Valid: 44 [ 150/390]  Loss: 1.958 (1.59)  Acc@1: 60.9375 (62.5931)  Acc@5: 87.5000 (88.0174)
Valid: 44 [ 200/390]  Loss: 1.896 (1.59)  Acc@1: 62.5000 (62.4145)  Acc@5: 82.8125 (87.6788)
Valid: 44 [ 250/390]  Loss: 1.524 (1.61)  Acc@1: 60.9375 (62.0020)  Acc@5: 89.0625 (87.5436)
Valid: 44 [ 300/390]  Loss: 1.479 (1.61)  Acc@1: 60.9375 (61.8978)  Acc@5: 92.1875 (87.6038)
Valid: 44 [ 350/390]  Loss: 1.268 (1.61)  Acc@1: 71.8750 (61.9881)  Acc@5: 85.9375 (87.6291)
Valid: 44 [ 390/390]  Loss: 2.003 (1.61)  Acc@1: 47.5000 (61.8960)  Acc@5: 82.5000 (87.5200)
valid_acc 61.896000
epoch = 44   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1086, 0.8914],
        [0.1858, 0.8142],
        [0.2106, 0.7894],
        [0.3899, 0.6101],
        [0.2479, 0.7521],
        [0.3053, 0.6947],
        [0.5029, 0.4971],
        [0.2809, 0.7191],
        [0.1038, 0.8962],
        [0.2526, 0.7474],
        [0.4890, 0.5110],
        [0.3300, 0.6700],
        [0.0758, 0.9242],
        [0.0321, 0.9679]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2756, 0.7244],
        [0.2146, 0.7854],
        [0.1823, 0.8177],
        [0.2479, 0.7521],
        [0.2067, 0.7933],
        [0.3543, 0.6457],
        [0.1610, 0.8390],
        [0.2193, 0.7807],
        [0.2782, 0.7218],
        [0.4120, 0.5880],
        [0.2989, 0.7011],
        [0.2168, 0.7832],
        [0.3191, 0.6809],
        [0.2616, 0.7384]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 45 [   0/390]  Loss: 0.3083 (0.308)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 1.587e-03
Train: 45 [  50/390]  Loss: 0.1849 (0.243)  Acc@1: 93.7500 (93.1985)  Acc@5: 100.0000 (99.6017)LR: 1.587e-03
Train: 45 [ 100/390]  Loss: 0.3173 (0.248)  Acc@1: 92.1875 (92.9146)  Acc@5: 100.0000 (99.6132)LR: 1.587e-03
Train: 45 [ 150/390]  Loss: 0.4328 (0.256)  Acc@1: 87.5000 (92.5497)  Acc@5: 100.0000 (99.6275)LR: 1.587e-03
Train: 45 [ 200/390]  Loss: 0.3541 (0.261)  Acc@1: 89.0625 (92.3818)  Acc@5: 98.4375 (99.6580)LR: 1.587e-03
Train: 45 [ 250/390]  Loss: 0.2368 (0.261)  Acc@1: 90.6250 (92.4178)  Acc@5: 100.0000 (99.6389)LR: 1.587e-03
Train: 45 [ 300/390]  Loss: 0.2217 (0.260)  Acc@1: 90.6250 (92.4938)  Acc@5: 100.0000 (99.6366)LR: 1.587e-03
Train: 45 [ 350/390]  Loss: 0.1548 (0.263)  Acc@1: 96.8750 (92.3834)  Acc@5: 100.0000 (99.6216)LR: 1.587e-03
Train: 45 [ 390/390]  Loss: 0.2871 (0.263)  Acc@1: 90.0000 (92.3320)  Acc@5: 100.0000 (99.6320)LR: 1.587e-03
train_acc 92.332000
Valid: 45 [   0/390]  Loss: 1.653 (1.65)  Acc@1: 57.8125 (57.8125)  Acc@5: 87.5000 (87.5000)
Valid: 45 [  50/390]  Loss: 1.670 (1.62)  Acc@1: 54.6875 (61.3358)  Acc@5: 79.6875 (86.4890)
Valid: 45 [ 100/390]  Loss: 1.643 (1.66)  Acc@1: 56.2500 (61.0613)  Acc@5: 84.3750 (86.6491)
Valid: 45 [ 150/390]  Loss: 1.432 (1.65)  Acc@1: 57.8125 (61.2479)  Acc@5: 93.7500 (86.6618)
Valid: 45 [ 200/390]  Loss: 1.668 (1.65)  Acc@1: 60.9375 (61.3884)  Acc@5: 82.8125 (86.6838)
Valid: 45 [ 250/390]  Loss: 1.602 (1.66)  Acc@1: 60.9375 (61.4231)  Acc@5: 85.9375 (86.8215)
Valid: 45 [ 300/390]  Loss: 1.392 (1.65)  Acc@1: 70.3125 (61.6175)  Acc@5: 89.0625 (86.9809)
Valid: 45 [ 350/390]  Loss: 2.131 (1.64)  Acc@1: 53.1250 (61.7388)  Acc@5: 84.3750 (87.1528)
Valid: 45 [ 390/390]  Loss: 1.094 (1.64)  Acc@1: 72.5000 (61.7120)  Acc@5: 87.5000 (87.2160)
valid_acc 61.712000
epoch = 45   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1047, 0.8953],
        [0.1837, 0.8163],
        [0.2095, 0.7905],
        [0.3929, 0.6071],
        [0.2493, 0.7507],
        [0.3030, 0.6970],
        [0.5054, 0.4946],
        [0.2795, 0.7205],
        [0.1022, 0.8978],
        [0.2524, 0.7476],
        [0.4899, 0.5101],
        [0.3291, 0.6709],
        [0.0733, 0.9267],
        [0.0308, 0.9692]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2723, 0.7277],
        [0.2128, 0.7872],
        [0.1790, 0.8210],
        [0.2421, 0.7579],
        [0.2030, 0.7970],
        [0.3540, 0.6460],
        [0.1547, 0.8453],
        [0.2131, 0.7869],
        [0.2785, 0.7215],
        [0.4086, 0.5914],
        [0.2991, 0.7009],
        [0.2126, 0.7874],
        [0.3176, 0.6824],
        [0.2587, 0.7413]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 46 [   0/390]  Loss: 0.1762 (0.176)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 1.377e-03
Train: 46 [  50/390]  Loss: 0.2068 (0.246)  Acc@1: 95.3125 (93.2904)  Acc@5: 100.0000 (99.7243)LR: 1.377e-03
Train: 46 [ 100/390]  Loss: 0.2266 (0.244)  Acc@1: 93.7500 (93.1002)  Acc@5: 100.0000 (99.7679)LR: 1.377e-03
Train: 46 [ 150/390]  Loss: 0.3581 (0.245)  Acc@1: 93.7500 (93.0567)  Acc@5: 100.0000 (99.7103)LR: 1.377e-03
Train: 46 [ 200/390]  Loss: 0.3042 (0.247)  Acc@1: 90.6250 (93.1048)  Acc@5: 100.0000 (99.6657)LR: 1.377e-03
Train: 46 [ 250/390]  Loss: 0.4068 (0.246)  Acc@1: 85.9375 (93.0652)  Acc@5: 100.0000 (99.7012)LR: 1.377e-03
Train: 46 [ 300/390]  Loss: 0.2178 (0.245)  Acc@1: 98.4375 (93.0804)  Acc@5: 100.0000 (99.7197)LR: 1.377e-03
Train: 46 [ 350/390]  Loss: 0.2795 (0.244)  Acc@1: 87.5000 (93.0689)  Acc@5: 100.0000 (99.7285)LR: 1.377e-03
Train: 46 [ 390/390]  Loss: 0.2560 (0.246)  Acc@1: 95.0000 (93.0480)  Acc@5: 97.5000 (99.6960)LR: 1.377e-03
train_acc 93.048000
Valid: 46 [   0/390]  Loss: 1.354 (1.35)  Acc@1: 64.0625 (64.0625)  Acc@5: 90.6250 (90.6250)
Valid: 46 [  50/390]  Loss: 2.110 (1.65)  Acc@1: 65.6250 (61.7953)  Acc@5: 79.6875 (86.9792)
Valid: 46 [ 100/390]  Loss: 1.227 (1.64)  Acc@1: 71.8750 (62.0204)  Acc@5: 93.7500 (87.3762)
Valid: 46 [ 150/390]  Loss: 1.947 (1.63)  Acc@1: 60.9375 (61.9826)  Acc@5: 82.8125 (87.2103)
Valid: 46 [ 200/390]  Loss: 1.816 (1.62)  Acc@1: 65.6250 (62.0336)  Acc@5: 85.9375 (87.3134)
Valid: 46 [ 250/390]  Loss: 1.457 (1.62)  Acc@1: 67.1875 (62.0020)  Acc@5: 89.0625 (87.4377)
Valid: 46 [ 300/390]  Loss: 1.444 (1.62)  Acc@1: 64.0625 (62.0328)  Acc@5: 87.5000 (87.3858)
Valid: 46 [ 350/390]  Loss: 1.609 (1.62)  Acc@1: 53.1250 (62.0370)  Acc@5: 89.0625 (87.4332)
Valid: 46 [ 390/390]  Loss: 1.132 (1.62)  Acc@1: 60.0000 (61.9680)  Acc@5: 97.5000 (87.4440)
valid_acc 61.968000
epoch = 46   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1007, 0.8993],
        [0.1812, 0.8188],
        [0.2087, 0.7913],
        [0.3949, 0.6051],
        [0.2490, 0.7510],
        [0.2995, 0.7005],
        [0.5106, 0.4894],
        [0.2801, 0.7199],
        [0.1010, 0.8990],
        [0.2502, 0.7498],
        [0.4924, 0.5076],
        [0.3317, 0.6683],
        [0.0718, 0.9282],
        [0.0299, 0.9701]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2705, 0.7295],
        [0.2131, 0.7869],
        [0.1763, 0.8237],
        [0.2348, 0.7652],
        [0.2005, 0.7995],
        [0.3515, 0.6485],
        [0.1516, 0.8484],
        [0.2083, 0.7917],
        [0.2748, 0.7252],
        [0.4034, 0.5966],
        [0.2940, 0.7060],
        [0.2083, 0.7917],
        [0.3110, 0.6890],
        [0.2575, 0.7425]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 47 [   0/390]  Loss: 0.1355 (0.135)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [  50/390]  Loss: 0.1853 (0.207)  Acc@1: 93.7500 (94.4547)  Acc@5: 100.0000 (99.8468)LR: 1.213e-03
Train: 47 [ 100/390]  Loss: 0.1703 (0.217)  Acc@1: 93.7500 (94.1522)  Acc@5: 100.0000 (99.8608)LR: 1.213e-03
Train: 47 [ 150/390]  Loss: 0.2993 (0.224)  Acc@1: 92.1875 (93.8949)  Acc@5: 98.4375 (99.7930)LR: 1.213e-03
Train: 47 [ 200/390]  Loss: 0.2802 (0.227)  Acc@1: 93.7500 (93.7267)  Acc@5: 98.4375 (99.7435)LR: 1.213e-03
Train: 47 [ 250/390]  Loss: 0.1639 (0.234)  Acc@1: 96.8750 (93.5819)  Acc@5: 100.0000 (99.7012)LR: 1.213e-03
Train: 47 [ 300/390]  Loss: 0.1526 (0.235)  Acc@1: 96.8750 (93.5995)  Acc@5: 100.0000 (99.7093)LR: 1.213e-03
Train: 47 [ 350/390]  Loss: 0.3543 (0.238)  Acc@1: 90.6250 (93.3894)  Acc@5: 100.0000 (99.7151)LR: 1.213e-03
Train: 47 [ 390/390]  Loss: 0.1827 (0.238)  Acc@1: 95.0000 (93.3760)  Acc@5: 100.0000 (99.7080)LR: 1.213e-03
train_acc 93.376000
Valid: 47 [   0/390]  Loss: 2.161 (2.16)  Acc@1: 53.1250 (53.1250)  Acc@5: 82.8125 (82.8125)
Valid: 47 [  50/390]  Loss: 1.923 (1.71)  Acc@1: 57.8125 (60.8762)  Acc@5: 84.3750 (85.7843)
Valid: 47 [ 100/390]  Loss: 1.590 (1.70)  Acc@1: 65.6250 (61.5718)  Acc@5: 87.5000 (86.0922)
Valid: 47 [ 150/390]  Loss: 1.543 (1.69)  Acc@1: 62.5000 (61.4342)  Acc@5: 87.5000 (86.4549)
Valid: 47 [ 200/390]  Loss: 1.891 (1.68)  Acc@1: 53.1250 (61.4350)  Acc@5: 85.9375 (86.6993)
Valid: 47 [ 250/390]  Loss: 1.380 (1.66)  Acc@1: 67.1875 (61.7343)  Acc@5: 89.0625 (87.0705)
Valid: 47 [ 300/390]  Loss: 1.249 (1.64)  Acc@1: 64.0625 (61.8875)  Acc@5: 90.6250 (87.2768)
Valid: 47 [ 350/390]  Loss: 1.841 (1.65)  Acc@1: 59.3750 (61.8100)  Acc@5: 84.3750 (87.1884)
Valid: 47 [ 390/390]  Loss: 0.8839 (1.64)  Acc@1: 77.5000 (61.9000)  Acc@5: 95.0000 (87.1720)
valid_acc 61.900000
epoch = 47   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0967, 0.9033],
        [0.1796, 0.8204],
        [0.2080, 0.7920],
        [0.3925, 0.6075],
        [0.2502, 0.7498],
        [0.3001, 0.6999],
        [0.5104, 0.4896],
        [0.2820, 0.7180],
        [0.0998, 0.9002],
        [0.2483, 0.7517],
        [0.4907, 0.5093],
        [0.3306, 0.6694],
        [0.0704, 0.9296],
        [0.0291, 0.9709]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2688, 0.7312],
        [0.2088, 0.7912],
        [0.1708, 0.8292],
        [0.2334, 0.7666],
        [0.1988, 0.8012],
        [0.3526, 0.6474],
        [0.1481, 0.8519],
        [0.2077, 0.7923],
        [0.2745, 0.7255],
        [0.4046, 0.5954],
        [0.2896, 0.7104],
        [0.2068, 0.7932],
        [0.3102, 0.6898],
        [0.2559, 0.7441]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 48 [   0/390]  Loss: 0.1160 (0.116)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [  50/390]  Loss: 0.1662 (0.212)  Acc@1: 95.3125 (94.6078)  Acc@5: 100.0000 (99.7549)LR: 1.095e-03
Train: 48 [ 100/390]  Loss: 0.1863 (0.213)  Acc@1: 96.8750 (94.5080)  Acc@5: 100.0000 (99.7679)LR: 1.095e-03
Train: 48 [ 150/390]  Loss: 0.1813 (0.216)  Acc@1: 96.8750 (94.4536)  Acc@5: 100.0000 (99.7517)LR: 1.095e-03
Train: 48 [ 200/390]  Loss: 0.1903 (0.220)  Acc@1: 96.8750 (94.1542)  Acc@5: 100.0000 (99.7823)LR: 1.095e-03
Train: 48 [ 250/390]  Loss: 0.2524 (0.220)  Acc@1: 93.7500 (94.0613)  Acc@5: 100.0000 (99.7883)LR: 1.095e-03
Train: 48 [ 300/390]  Loss: 0.2501 (0.225)  Acc@1: 90.6250 (93.9265)  Acc@5: 100.0000 (99.7664)LR: 1.095e-03
Train: 48 [ 350/390]  Loss: 0.3131 (0.226)  Acc@1: 93.7500 (93.8969)  Acc@5: 98.4375 (99.7730)LR: 1.095e-03
Train: 48 [ 390/390]  Loss: 0.3175 (0.227)  Acc@1: 90.0000 (93.8120)  Acc@5: 100.0000 (99.7800)LR: 1.095e-03
train_acc 93.812000
Valid: 48 [   0/390]  Loss: 1.462 (1.46)  Acc@1: 68.7500 (68.7500)  Acc@5: 89.0625 (89.0625)
Valid: 48 [  50/390]  Loss: 1.695 (1.67)  Acc@1: 64.0625 (62.0404)  Acc@5: 89.0625 (86.9485)
Valid: 48 [ 100/390]  Loss: 1.416 (1.67)  Acc@1: 71.8750 (61.4171)  Acc@5: 90.6250 (87.2834)
Valid: 48 [ 150/390]  Loss: 1.539 (1.65)  Acc@1: 68.7500 (61.8998)  Acc@5: 85.9375 (87.3241)
Valid: 48 [ 200/390]  Loss: 1.529 (1.67)  Acc@1: 65.6250 (61.6915)  Acc@5: 87.5000 (87.2357)
Valid: 48 [ 250/390]  Loss: 1.657 (1.66)  Acc@1: 57.8125 (61.6721)  Acc@5: 90.6250 (87.1327)
Valid: 48 [ 300/390]  Loss: 2.250 (1.65)  Acc@1: 57.8125 (61.8148)  Acc@5: 85.9375 (87.3339)
Valid: 48 [ 350/390]  Loss: 2.042 (1.64)  Acc@1: 57.8125 (61.9658)  Acc@5: 78.1250 (87.3086)
Valid: 48 [ 390/390]  Loss: 2.532 (1.64)  Acc@1: 45.0000 (62.0000)  Acc@5: 72.5000 (87.2880)
valid_acc 62.000000
epoch = 48   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0936, 0.9064],
        [0.1789, 0.8211],
        [0.2051, 0.7949],
        [0.3943, 0.6057],
        [0.2502, 0.7498],
        [0.2980, 0.7020],
        [0.5074, 0.4926],
        [0.2851, 0.7149],
        [0.0993, 0.9007],
        [0.2503, 0.7497],
        [0.4922, 0.5078],
        [0.3348, 0.6652],
        [0.0706, 0.9294],
        [0.0286, 0.9714]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2645, 0.7355],
        [0.2063, 0.7937],
        [0.1670, 0.8330],
        [0.2287, 0.7713],
        [0.1973, 0.8027],
        [0.3520, 0.6480],
        [0.1439, 0.8561],
        [0.2081, 0.7919],
        [0.2723, 0.7277],
        [0.4071, 0.5929],
        [0.2880, 0.7120],
        [0.2063, 0.7937],
        [0.3060, 0.6940],
        [0.2562, 0.7438]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 49 [   0/390]  Loss: 0.2719 (0.272)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [  50/390]  Loss: 0.1607 (0.206)  Acc@1: 96.8750 (94.6998)  Acc@5: 100.0000 (99.8775)LR: 1.024e-03
Train: 49 [ 100/390]  Loss: 0.2668 (0.205)  Acc@1: 89.0625 (94.8484)  Acc@5: 100.0000 (99.8298)LR: 1.024e-03
Train: 49 [ 150/390]  Loss: 0.3559 (0.217)  Acc@1: 90.6250 (94.1846)  Acc@5: 100.0000 (99.8241)LR: 1.024e-03
Train: 49 [ 200/390]  Loss: 0.2277 (0.222)  Acc@1: 93.7500 (93.9055)  Acc@5: 100.0000 (99.8134)LR: 1.024e-03
Train: 49 [ 250/390]  Loss: 0.1386 (0.223)  Acc@1: 95.3125 (93.9119)  Acc@5: 100.0000 (99.8070)LR: 1.024e-03
Train: 49 [ 300/390]  Loss: 0.2925 (0.223)  Acc@1: 92.1875 (93.8227)  Acc@5: 98.4375 (99.7872)LR: 1.024e-03
Train: 49 [ 350/390]  Loss: 0.2648 (0.224)  Acc@1: 93.7500 (93.8034)  Acc@5: 100.0000 (99.7819)LR: 1.024e-03
Train: 49 [ 390/390]  Loss: 0.2211 (0.224)  Acc@1: 95.0000 (93.8160)  Acc@5: 100.0000 (99.7840)LR: 1.024e-03
train_acc 93.816000
Valid: 49 [   0/390]  Loss: 0.9788 (0.979)  Acc@1: 70.3125 (70.3125)  Acc@5: 95.3125 (95.3125)
Valid: 49 [  50/390]  Loss: 1.921 (1.58)  Acc@1: 59.3750 (61.9792)  Acc@5: 87.5000 (88.4191)
Valid: 49 [ 100/390]  Loss: 1.513 (1.63)  Acc@1: 62.5000 (61.2160)  Acc@5: 87.5000 (87.7475)
Valid: 49 [ 150/390]  Loss: 1.220 (1.63)  Acc@1: 65.6250 (61.3928)  Acc@5: 89.0625 (87.6966)
Valid: 49 [ 200/390]  Loss: 2.446 (1.64)  Acc@1: 50.0000 (61.3650)  Acc@5: 78.1250 (87.7177)
Valid: 49 [ 250/390]  Loss: 1.073 (1.63)  Acc@1: 67.1875 (61.5040)  Acc@5: 92.1875 (87.5623)
Valid: 49 [ 300/390]  Loss: 1.512 (1.64)  Acc@1: 67.1875 (61.5033)  Acc@5: 89.0625 (87.4689)
Valid: 49 [ 350/390]  Loss: 1.951 (1.65)  Acc@1: 56.2500 (61.3782)  Acc@5: 82.8125 (87.4110)
Valid: 49 [ 390/390]  Loss: 2.063 (1.65)  Acc@1: 52.5000 (61.4720)  Acc@5: 87.5000 (87.4640)
valid_acc 61.472000
epoch = 49   
 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4), ('sep_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_3x3', 1), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0909, 0.9091],
        [0.1808, 0.8192],
        [0.2059, 0.7941],
        [0.3956, 0.6044],
        [0.2499, 0.7501],
        [0.2958, 0.7042],
        [0.5128, 0.4872],
        [0.2848, 0.7152],
        [0.1007, 0.8993],
        [0.2479, 0.7521],
        [0.4946, 0.5054],
        [0.3330, 0.6670],
        [0.0707, 0.9293],
        [0.0279, 0.9721]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.2616, 0.7384],
        [0.2062, 0.7938],
        [0.1654, 0.8346],
        [0.2258, 0.7742],
        [0.1952, 0.8048],
        [0.3493, 0.6507],
        [0.1405, 0.8595],
        [0.2081, 0.7919],
        [0.2658, 0.7342],
        [0.4089, 0.5911],
        [0.2850, 0.7150],
        [0.2041, 0.7959],
        [0.2971, 0.7029],
        [0.2527, 0.7473]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
